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ABSTRACT , ' f ^ • . . ' 

The papers included in , this collection represent as . 
closely as" possible the content atod organisation of the oral 
presentations delivered at a tfttrch 1980 conference on the use of 
ceadaSility formulas. The papers discuss the following topics: (1} an 
introduction to the conference,/ with general remarks 02} the uses and 

'celticism" of readability formulas; (2) €he iaevelopmeut of readability 
forprula^s and attendant problems in the validation of readability 
formulas; (3) readability formula and the definition of the' task of 
reading; (4) cases yhere readability 1 formulas do hot work well, 
including/ television captions for the deaf, elementary school- 
tradebooks, reading qom prehension tests, remedial reading tfsx£s,.ana 

.basal readers; {5) lowering the reading difficulty level* of t &tts 
intended for adults, with implications for "pla^a/langtia^" in legal 
documents; (6) readability' formulas <and the adaptatioas/ojt texts r. 17) 
" the comprehension of captioned television; and ,(B) arguments against 
some uses .of readability* formulas, with suggested alternatives to 
r^idfibility ^ormtflas* / <Ri)\ . • 9 
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' m Foreword 

,The papers which make up this Technical Report are summaries of oral* 

\ ; 

presentations on readability and readabi 1 Ity ^formulas del i vef=ed~on-Harctr ifr 



and h, 1980 at the Center for the Study pf jteading. There is some overlap 

» * 1 

• • - \ I * ' - 

in ,the material covered in different presentations , as our organization 

of the sections reflects the content of the particular parper presented t^y 

the authors named" on it. Those who spoke ^re listed as authors. AcknowJ- 



edgment is given to co-authors in the form of footnotes. Host of, the papers 
were prepared from texts Submitted by the authors, and all of them have- been 
read and corrected by the authors. Several of the summaries of papers and 
aJT of the summaries of discussion were prepared frdfn tape recordings of 
the procedures by graduate student members of the Text Analysis Group, 
Center for the Study of Reading. These^ incl uded Jean Hannah, Hacgaret Laff,, 



and Robert Salzillo* * * 

The papers incl uded ^frgr e represent as closely as possible the content 
and organization of the oral presentations, in a more readable format 
than a verbatim transcript. We would like to emphasize that the purpose 

* m 
* 1 

of the conference was £o rafse a number -of issues for discussion and Co 
pre^nt a spectrum of ideas and viewpoints from which readability formulas 
could be Judged or criticized. We have nQt tried to make 'the papers 
exhaustive summaries of all -that has'been dbne on a certalnfsubject or 
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to represent ortly. the mos^t "correct" and orthodox positions on any" topic. 
We feel that the views expressed" here , while diverse and in some cases / 
programmatic, will be useful in provoking discuss ion, and reexamining 
assumptions abdut readabi 1 1 ty formulas, perhaps in defining research which 
might lead to a better understanding of what makes things difficult to read 

1- introduction to tb£ Conference — AJjce Davison 
General Remarks 

this introduction, \ want to desyibe how we came to consider the 
questions raised and discAJs^ed during the conference. Basical.ly our 
interest grew out of research which was originaMy not directed toward 
the criticism of. readabi 1 ity ^fortoul as, a- study done at thp Center for 
the Study of Reading by the linguists in the Text Analysis Group. "In 

' " * ' V 

this study, wadset out to analyze the structures of te-xts used as reading 

* 

.materials for children. " In doing this, wfe hit upon the idea of comparing 
the versjon of a text used for this purpose wi tfAie original text from 
Which i tt had been adapted, which might involve a change in structure and 
also some simplification to make the adapted text easier to, read. Somewhat 
to our surprise, we found th/t> the changes made irt adaptation were of very 
great interest, and our study was focused on characterizing the kinds of 
changes made and the"possibJe motivation behind them, assuming that any; 



chartges made were Intended to make the text easier~4o read. , #s> 

> 
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■ ' ■* 

The results are discussed in Section 6 by Roberf Kantor and Alice 

Davison, but here I want to note oneofo.ur conclusions, based on the 

following points. Words in the text which -were not necessarily obscurT^^J 

In context , or even very infrequent, were changed inrfadaptation to shorter 

words or roore^ familiar words. Changes made in sentence structure , such 

as splitting a long sentence into several sentences , tended to make 

sentences shorter throughout the text and in the overall average sentence 

length. Such changes were made even when the original sentence structure' 

did not seem to us to. present any particular difficulties. The commonly 

used readability formulas measure vocabulary frequency or number of syllables 

per N word, plus sentence length — exactly the features of the texts Vdiieh 

t " / 

were modified without apparent motivation within the text. It seems that 

the. changes were made tn order to make the adapted texts, "meet" a certain 

level of reading ski IT defined by these formulas. * 

Other kinds of changes'were made in order to'clarify or simplify 

the te/ts . From these, we were able to isolate some other features 

of texts, less easi ly^neasurable by formulas, which ^n oOr view 

contributed to text comprehension as muph as, or rtore than, sentence 

length, vocabulary complexity (see* Section 6, Davison and Kantor). 

In fapt, some of the cases where vocabulary was simplified or sentences 

were split up ten Jed to make the .text less easy to comprehend, because 

the relations between the parts of the text were less clearly expressed. 



• 9 ' . . TO - %j 

ERJC- - . - ■ • 



Readability Conference 

t 



\ m • 

One of the main conclusions 6f our text analyst* study was that 

readability formulas, whatever their valuq as predictive, statistically 

<? • * • 

based measures, are'no^ particularly helpful in directing writing or 
adaptation. They fail to address the central issue: "What is it^out 
the language of a text which might normally make it easy or hard for 
a reader to understartd the text? (See von Glaserfeld, fe70-7l , for' a 
lucid discussion of what might be meant by Syntactic complexity fcnd 
how it-affects comprehension,) * 4 

There are increasing nuiribers of, cases where the answers to th(^ 
question are necessary/' If the language of some texts is such that 
people who need to be able to understand the texts' are f in some large, ' 
numbers, unable' to do so, then it would be. useful to t>ave some set of 

principle^ for changing the language of* the text to make i t m comprehe/is * b T 

i 

to people reading at some specific level of- reading achievement. Here 
a systematic ambiguity in the term readability becomes apparent. Some- 



times the term is used to refer to text types or styles of writing 

+ 



thfat 

people find attractive and hence easier to rea<J than other, kinds of 

fc ^ 1 ) * 

writing (e.g., Flesch, 19^8) « fJut nowadays the main concern is with 

what people are able, to read, under normal circumstances. 

The problems of major concern today have to do mainly with texts 1 
♦ * '< < x ' < v - . 

that, people must read and be able to 'understand, Such as school and 

college-level textbooks, tax forms and instructions, legal documents 

and simitar texts on technical subjects whose contents must be clear 1p 

expressed in language understandable to the general public. Of course, 

not all of tKese ajreffs of concern are directly related to/one another. 
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Some Important differences having to do with tasks and with child versusf 
# * * 

x adult readers are <Jiscu$sed in the following sections. Nevertheless, 
these £re all areas where wrFters.and publishers have looked to readabi 1 i ty 

* • ■ - 

formulas for some sort of guidance. - V* 

* • . . i - C 

^ • . • ' ■ " • v ■ . 

. Uses for Readability Formulas .„ ■ 
i — ' - • 

Formulas were originally devised as predictive averages which would 

be useful in ranking in order of difficulty a number of books for use in 

a particular school grade* They were to be simple enough to be applied , 

by . people without extensile training, and to be applicable to texts of. 

many different types. Thus, as the creators of formulas have always* 

pointed out, the formulas were intended "for certain specif ic \uses , for 

example, estimating the difficulty of a text In reflation to some other text- 

'It seems clear that formulas, which measure objectively definable features 

such as sentence length and more or Jess definable features like vocabulary 
• m • * *■ 

complexity, may reflect but do not characterize the real 'factors which 

/ . - * f 

contribute to ease or difficulty in heading a text. Formulas therefore, 

annot serve as guides for writing^ nor were thpy meant to. Proponents 

of formulas Such as KJare M963, 1974-75) point out that ttie use gf formulas' 

presupposes the existence of a well-formed text to begin with, one which 

has been put together by a writer using whatever common sense and writing 

skill he* or she possesses, Including the ability to present a subject 

coherently, etc. The vdlid use of formulas thus; ^presupposes a number of 



* • v ; • ~ f 
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* £»prpperties of text besides sentence length arid yocabuldry complexity, and 

• the'se^pfopert les a re Undefined unexamined b^Hme author! of formulas. 

% ' * ■ * . v ' . / , • , 

Although fojWl/s were rfot designed to Jbe.used 1 as jgui des /for writing, 

I , they are used Jn this way, though ^obody really Ifciows- howwidely . 4 Evert 

^j^not; actually used as guidelines v they have the in^Jinjtt effect of 

^guidelines if {hey $re used to define a &t^ndar*d reading* Ipvel which a. 

, given text must meet. Publishers and writers who mu^t create riding' 

materials at levels of 'reading difficulty defined by formulas are under 

greajt pressure to make their matari&B conform by altering vocabulary 

.^hoices and sentence length. ^ * • 



v - ♦ » « • > i 

E«liec Criticism of Formulas ,* 

* At the end^f the pdaptSt io^^tudy^vhi ch compared t^wo versions <?f 

the same text, w/felt a glow of^hoapst pnele* about % what we had discovered 

reg t ar4ing changes in adaptations and, about our conclusions, which were 

critical of the use of readability formulas, Then we discovered that for 

at least the past te^ ; years, a number of #ther peopJe h^ve comer tp> the 

very same conclusions about formulas. (Bote\ S Grancwsky, 1972; Dawklqs, 

0* 197^; 'Endhcott, 1973; Courley # $ Cafllnl 1978; Nelson^ 19784 Reddin, 1970, 

Selden, )977;?von.Glaser'feld, ,1970-71; Fagan, Notfe I; Schmidt, Hote~ # 2), 

-and tjave proposed various alternative measurements k mostly in very / 

* % * • 

* . 1 - . 

, pro^ammatic form-, TJ>e common denominator of these proposals ts some, 

* • 1 
<• 1 * ' * 

^method of measuring syntactic complexity more directly* than juft 
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* " " ' - ' 

through sentence Jength.. Such measures at least make the measurement 

* . r 4 « j» * 

of complexity more text specific, although they, do ifot attempt to 
characterize such larger notions as presentation i of Ideas , definition 
of topic, etc. • r 

\ m • 

Vfe were surprised that the articles which preceded our work had not 

had very, much effect on the way that readabll 1 formulas are - perceived. 

/ * 
Formulas 'continue 'to b& In widespread* use/ both as measures and guide 

♦ / 

lines, In spite of awareness on the part of some researchers- of their 
very serious drawbacks' as. reliable Instruments. One probable* reason 
for this Is that formalas have very.strong economic advantages In that - 

* \ ' • c 

f • * 

they can be used without special equipment by people without special 
training. The npre direct measures of syntactic complexity and other, 
features require some sped a 1J zed knowledge of language structures on 
the part of the analyst apd probably, take more time to apply because the 
text^usit be very, closely ejj j^^fc nd analyzed; Readability formulas 
also have the advantage of.be*Rtj well known, fami 1 iar ,*and already part 
of the scheme of thtng£, while the alternatives are speculative and 
relatively untried. x • / ^ 

Nevertheless, the disadvantages of using traditional readability 

i + # • • 

formulas, particularly for 'revising or writing texts, seem qbvlou? 

and compel 1 Ing to anyone who seriously considers them. Much energy 

* * r i 

has b^en Invested In refining and simplifying formulas, with fewer or 

more' easily defined , variables (see surveys such as l^are, 19<>3, 197^- 

9 > . • * - • * * 

75). Not^ much attention seems to have been paid to the mudh more 
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central and interesting questions of what makes a text difficult to 
process and comprehend , and exactly how the language in a text 
contributes to difficulty. ' t , 

Focus of t he Conference 

1/ the answers to v thes? questions were found, there might havfe beeln 

*' 9 ■ • 
a number of political ^and economic consequences* But leaving aside 

such considerations, we approach formulas from two cri tical points of 

view in this conference, *ln order to establish what the intellectual 

.Issues are. first, to^what degree are readability formulas really 

adequate and justified as dSvicas t for measurement*-wha t is their 

* M V ' • ■ 

conceptual basis* how were they initially validated, to what uses 
can they legitimately be putf? Where they fail .to be* adequate *or 
appropriate, what are valid and compelling arguments against ,them? 

Second, -what alternatives are.. there for readability formulas,, 
especially for particular purposes, such as writing and revising texts? 
Some proposals have Included taxonomies of structure types (Bote<*& 

Granowsky, 1972; Dawklns, 1 975) » invest I gallons of reading as a left 

• • 9 

s to fight -parsing of_sentences*or longer structures (Richek, 1976; , 

von Glaserfeld, 1970-71 ;> Fagan, Mote 1), weighting of dlffecent general 

r \ • \ 

structures (Endl^ott, 1973;\ Reddin p 1970; Schmidt, Note 2; Selden,' 
Note 2), measurement of difficulty rc^latlye^ft ^ntext In the discourse. 
(Gourlay, 1978; Gourlay l <ktl in,* 1978), ancTsotffV' What would tonstl- 
t'ute compel 1 InSr evidence fey such descriptions as improvements on 



readability formulas? /How co'uld these, altprnaftvasJ^e shown to be better 
guides to writing £hari formulas if re? p 

/please note 4 that the presentations at this conference Are critical 
of readability formulas. This 'drjxntatioo Is Intentional; we feel that 

V T ' \ 

readability formulas have had a Kidfc hearing throughout their hi sloty * \ 
ahd that they do-not need dpferfders nere simply for the sake o f ba lance* 
But our ^urpose^ is rx^t *to- condemn readability formulas out of hand. 
. Rather, it is ^to clarify *the f'ssues* surrounding readability formulas 
and to define the logi^c behind Various views. Often it seems that 
defenders of formulas and, critics of formulas are r^Uy tSlWng 
- about" separate issues, for example* statistically valid predictive 
Vi averages versus explicit, internally consistent models of language , 

processing, It is important* to^pjace both cri ti-cisms. and defenses 

. »» » 

within the right contexts of-discourse and not to generalize from 
» * * - 

one domain of discourse to another. The outcome or .dfscussion can - 

• * » 

have no value^Jf the issues are not" debated ^ithin clearly defined 



* contexts , 
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All of the peopl£\who spoke at the conference have had to confront 
,* readability formulas in sope area where it is important to iftake a text 
comprehensible to a particular audience ^vre^ers . With the pofnts 

that they raised, and # the ^fscujssidnof th*em,*we hope to make some ^ 

. } ' - ' " ■ 

. • .useful definitions of the Issues and arguments bearing on^thefla. 

* ■ • * 

t * ■ * * 

1 . < . . . ' 

. , - . 2. 4 On the Validation bf the Original Readability Formulas 

** ' ./ t « 1 

i *- • I . — 

* ■ x Ramsay. Selden " 

This, talk focuses on: (a) a brief history of • the development of % ,,. 

readability formulas, wl,th a cri tical. look at how f^esearchers settled 

On the most widely # used variables; (b) a variety o^ problems and , 

cf rdCildritles existing in>the validation of readability formulas; , 

: > f « * ' * * ' ~ f 

and (c) Selden's own res^ar'ch pn readabl 1 i ty , .which represents a departure 
from traditional use of readability formulas*and <which serve% to point 
out what is wrong with readability formulas and how 6hey are used. 



I. 

\ 
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Bps! cally. readability* formulas involve empirlcaj counts of tftree 

* - • 1 * 

or four charactetistics of a text, whjch are multiplied by weighted 

* * ■ 

* • « X 

t • V * 

coef f icients, with the resultant numbers being summed to give an 
Index of the readability of a piec^ of text. A readability Index 

number for a gLverr text Is typically based on three samples of .100 

* ■ * 
words each from that tekt. Readability formulas are intended to be 

indices of the difficulty of a passage, but they were jiever intended 

to provide specifications of text characteristics tnat contribute to *" 

text difficulty. * * 

^ h * ^ The Development .of Readability Formulas 

Readability research probably began In earnest wl&i the publica- 
' * « % * > * _ - 

tlon in 1920 of E. L. Thorndikef's Teacher's Word Book. This was a 

. / . _ ? 

listing of 10,000 vocabulary items stratified by their frequency of 

; . '* • 

^ v appearance per million words, in several. coYpora of language data. 

It represents a systematic effort to provide a tool for the estimation 

if w * * • — rZ. 

- of the diff icuity* of words which canr assist in the teacher's Intro* 

• - * , 

■ % 

ductlon of appropriate materia) to students leafning to read. Syste- 

/ ' h - .- . • >' • - 

matic analysis of-te#t was not, new, tfi course; for instance, Klare 

♦0963) cites studies 4n Jhe* 1800s aimed at classifying text by its era 
» ■ ■ 

* of^publ ication' acqprdf^g to*"the mecyi , length of its sentences. Prior % 
* to that, Herbert* Spenser and language philosophers had speculated on 

the characteristics of text £hat entered fnEo- readability and on the 

* ' * f % * \ * 

concfept of readability Itself .\ 3ti 1 1 , Thorndike's wGrdllst reflects a - 
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perception 6f the pedagogical' usefulness of examining reading materials 
and validly estimating tfje difficulty with which they may.be read-,- 

Lively and Pressey (1923) published a study of vocabulary burden based 

\ ' 
on tbe, Xhorndifce tfordl ist^and found the mean frequency ^trata of words 

in text to.be associated with the grade level In which the textuaj 

materials were to be used. This was an important precedent, as that 

has literally been the paradigm for readability technology. Using his 

own f requency-stratTfTed wordlist, Dortch (1928) found that book* within 

a single grade varied widely in estimated reading difficulty. 

-Around 1930, a substantial amount, of experimentation was begun 

concerning the Variables used as predictors of readability. In 1928, 

Vocjel ana Uashburne predicted (he tested reading, abi 1 i Jy, of subjects^ 

who liked a book on the basis of, Its proportion of unique vyords to total 

.words (type/token ratio}, proportion of prepositions, proportion of 

mm ' * 

words not oh the Thorndike Ii$t, and the proportion of 'simple sentences, 

^* * » » • 

Host useful were the Thorndike measure of vocabulary difficulty and 

• \ ' * . 
the proportion of simple aenteqdes- Lewerenz, in six studies from 
/ * *^ 

^929 to 1939, found the difficulty of feXt associated with the initial 

letter of words in the text t H l H and ,r E n were hard, 'M," % ,r and "B M fc 

were easy), with the presence of W&rds of* Greco-Roman derivation, ; 
*' * * 

with high"f iWuertcy vocabulary, with the presence of" sensory words, 
and with" the presence, of polysyllabic words. ' * 



- - J- 
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* Generally* attention in # investigations from 1930 to the present 

, has split Into considerations of word characteristics and considerations _ 

_ of sentence characteristics as predictors of readability. There Jias 

been virtually jio work on the characteristics pf the message structure^ 
* ■ - t 

of texts until* the work of -H&er (1975) and Kintsch and Vipond (1979) 

In the last few years. Summary v/prks such as the one <fone Jjy Klare 

(1963) indicate a set of word features which have consistently been 

( sho**p t to tie useful in estimating readability: vocabulary familiarity, 
based'On a frequency-stratified word list developed from samples of , 
text, prose, letters, or other bodies oF~ written language; word length 
measured both in letters and, in syllables; and word spelling," 
specifically the presence of,certairf graphemes In' certain positions 
in words. As one notable exception, Rudolf Flesch has not included 

, vocabulary familiarity in his" reacfabt llty formulas; Instead, his 

19^8 worj^detafjs two ^ubscales, a "reading ease" scale based on word 

length and sentence length and a 'human Interest 11 scale based on the 
* ■ * • 

rtumb£r of personal references tn the text. . His later studies described 

X 

developments of the readability measures based on the proportion of 

affixed and abstract words {Flesch, 1951, 195*0. Flesch' s formulas are 

- : f . ■ . , . . 

based on the premise that words in the text whjch appeal directly to * ' 

, ■ ,* 

the 'reader's attention and which provide content whi ch is (concrete^ 
3S opposed to abstract wl M be more interesting and more easily read. 
However*, as Klare points out pp. 19-20), this premise probably j>m> 



v 



t 
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hofds only for certain, types of text, such as fiction. Across all types 



of text, including technical , narrative, and expository writing, the„ 
words hypothesized to be readable by Flescfi would often be unantici- 
pated and coufd be expected to disrupt* leading, making it more difficult. 
"Significantly, the Flesch "reading ease 1 ' formula, based only on ) 
sentence length and word length, was cited by Klare as the single most 
widely used formula. . - , 

Researchers have experimented with a number tf sentence characteristics 
ai determinants of readability: numbers of prepositional phrases, degree 

I 

6f embedding or subordination, presence of structures such as passive 
verb forms. In most studies these variables have shown relatively 
weak correlations with criteria of readability, beyond the ^prrelation 
provided by sentence lengtti alone. Since these sentence-structure ■* 
variables are highly correlated with sentence length (highly subordinated 
variables tend to be long) / tK&y have not Endured as predittors of 

• / • - , 

readability. - x ^ 

/ / 

1 f To summarize, since its beginning in the early 1920s, readability 

i * 

research has investigated a variety*"!?? textual features, but has most 
consistently preferred a yna 1 1 set, which has displayed enduring useful- * 
nes*s . Investigation also explored alternative methods for measuring 
criterion measures, predictor measures, and assigning weights t6 jfre- 
dictors in the formulas,* By, the mid*1950s t mo£t^formu)as were working 
with just tti^ee variables--vocabulary familiarity, word length, and 
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* sentence length. In most widely used formulass a combination of two 
of these three variables is used. 

There seem to be two.printfry reasons why readability formulas *. 

• * 

settled on these three measures. First, formulas containing just two 
•of the variables characteristically show correlations of .7 and .8 
with criterion readability scores (Dale & Chall, .19^8; Fry, 1 968t 
Spache, 1953}. Yhls indicates a good deal of predictive p6wer, from 
$0% to (>5% of the variance in criterion readability, and there is 
little increase In predictive power to be gained through the / incorpora- 
tion of additional variables • This is pointed <)ut in Seldfen (1977); 

4 • 

since readability formutts fail to account for syntax except for what 

may be incidentally measured by sentence length, this study attempted 

to add syntactic variables to the formulas. Thej-esearch tested 
v - 

whether or not pieces of texts containing sentences that use relatively 

, common surface atrjngs would be more readable than texts with sentences 

containing relative-ty uncommon of infrequent surface strings* When a 

~. , ■ 
measure of frequency of surface strings was added as a variable to 

n . ' ' * * 

another formula, it was fpund that thlj added significantly to the 

* • * * * * * * 

predictive powers of the formula^ for older' rea'ders dealing with more 

difficult texts, but did not have much effect for younger readers* Whll 

■ w 

it was vali'd to assume that tha variable was significant to some degree, 

• / * 

th6 question was," Under what conditions Is It important to account for 
it?' The addition of a variable- has to be weighed against the payoff. 
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Thfc^second reason why readability formulas settled on -those three 
basic measures Is that It has been desirable for readability formulas 
to be convenient to use, requiring counts of textual variables which 
-can be compiled by a classroom teacher or publishing house staff 
without a great deal of time and complicated tabulation. This was 
probably more true in the .past, before there were computerized text- 
analysis capabilities. Also, the desire for convenience depends on 
the premise that classroom teachers use readability formulas on some, 
widespread basis to evaluate texts and to match them up % wi th students. 
But^ i'n the only known study of the use of such formulas In the class- 
room, part of a survey done by Jean Chall and Edgar Dale in the mid- 
1950s, very little use of readability formulas by teachers was found\ . 
Indeed, it seems logically and operationally unreasonable for teachers 
to use readability formtllas. Good reading Specialists and readipg 
teachers tend to work with two vast cjomains of intuitivV nonempi rlcal, 
information in matching children and texts. One domain is knowledge 
about the* child, which goes far beyond standardized reading test 
scored to include {information on what kind of reading materials the « 
chi Id. has read In the past, what his or her interest mijght be., more 
detailed Indications of what his or her reading problems might be, 
and so on. A second domain includes the vast amount/of -infornlkion 
about reading materials which *goes beyond what readability focmula's 
tend to measure Reading specialists and^ reading tutors have a > 
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# . repertoire of books they juse wltb children, and* for )$f)I ch they know 
the approximator *Jif faculty revel, yhat the story is £t>out , basically 
whl e ch words in the fjooK a particular ^chi Id mighty need special help 
with, and so on. Al/I of that information is Infinitely more useful 
in matching the book with the child than a technolbgical assessment of 
the material compared with the child's reading test score. ^ 

• \ . / . ' ■ ' 

Some Wfdety/tJsed Formulas 

- 4 ■ ■ . * ' * • 

Dale!- and Chall (1948), Dolcb (19481, Fry (19681, Gunning ^(.1952) , 
Spache (J9531, and Sticht (.1,972, 1975) have de.vfsed formulas. b(Jsed on 
t"he three basic measures mentioned above* (.vocabulary lists, word length, 

■* » 

* * V "* * 

sentence length, or a. combinat ioh of two of these measures), and wfiich 
nave both properties of being easy to use and of bearing a high degree 
of indicated predictive power. 

I */ • * 

t Dale and JChaH * * 

V jComprehens ion * .1579 $ words'not *on Dal^-Chali list of, 3000 

cotnmon words) + .<j£96 (words/sentence) + 3.6365. 

, -U Gunning * • 

* ' . Readability Index - (piean sentence length + % words over two . 

\ syllabjesh\ 

Spaphe # 
* < * ** 

9 Publisher's grade •level « .141 (mean serttence length) + ,086 

(% words ,not on Dale list of 769 easy 
words) + .839/ * . 
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Dolch 

Publisher's grade level ■ average of: t^ble values for median 

sentence length, 90th percent! Ife 
sentencayfenjjth, and % words not on 
f * ■ Dolch IjKst of 1000 easy words. 
■ J 

Publishers grade level.* intersection of vaUies for sentence length 

id word length measured in syllables on 
^ /the Fry Readability Graphs 

FORCAST (Sticht*, 197& 
v 



Comprehension - 2D , nuri>e r of one-syllable words 



The Validation Profc'l 



em 



The problem of validating these formulas nefeds^to be examined irr 
• * 

closer* detail* Most formulas are validated against one of two criterion 
measures: ^publishers' assigned grade level for passages from reading* 
texts, jor £hfe n&rms for the McCal 1-Crabbs comprehension passages, * The ^ 
Dale-Chall formula was validated against the^HcCall -CrabTbs (1925a* 
1925b, 196)) comprehension norms; Gunning, against other readability 
formulas: Spache, Dolch, and Fry against publishers/ assigned graafc* 
Jevel; and FORCAST. (Sticht, against comprehension questions. 

O r 

Formulas are developed by conducting muitlpT^e regression analyses on 

& set of passages (e.g.., 50) , using a range of predictor variables 

• i 
and working out the best set .of weights and constants to estimate * , 

the cri terion. . „ • ■ 

TKe publishers' assigned grade level simply reflects where a 

passage occurs in a basal. seHes (usually 1-1 through 6.£j. In.the 
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past, passages were assigned to theftr various levels by convnon s6nse r 

, writer feedback V.teacher feedback, and other Information f w^ttf formulas 

*\ ' ' % ^ 

being used as a" check. Since most has^l sefies go only thrcftjdh level * 

6.2, the formulas are thert<fcased -on eleroefttary, passages, even though } 

* " # ^ "a \ 
they are suppose^ to be predlctlye yp *to adult readfng levels^ Besides 

C* problem, there are circularities^ that plague the validation of # 
ula^ against ^publishers' assigned, arable teve). Virst Is a peda- 
gogical circularity In ,that f \6 some extent, the skills that are ^ » 
emphasized in instruction of s'chodi children qorrespond wfft the kinds 
of text characteristics the formulas are geared tq measure/ The v 
Harris-Jatobson (1974) formuja probably goes tfce farthest in dbing 
this, v In its four-variable version measures sentence ieogth, 
word length, word familiarity Ijased o n frequency of use Jn basal', 
reader programs, and word "decQdabllity'^based on ph6neme -grapheme 
correspondences. Word f requenc^and~ decodaW 1 1 ty are both heavily 
taught in basal reader prdgrams. Oye^ time^we approach a technology 
that oh the one band teaches certain sktlls*and on /the othqr hand 
evaluate the demand /or those sktfls in texts as Indira tors" of the* 1 

readability of the t^t." Th^re fs af, danger that after a whMe what ^ 

c -r s * * 

is measured js, sjmply what J&.taugh£, tearing a partial relation to, real- 

world literacy. 

/ 

In using / 

publishers 1 assigned grade le#f as a criterion. Simply, publishers use 



There is a .more superficial iev&i of ctrculjarlty Involved In us 
publishers' assigned g^ade leV^r as a criterion. Simpty, publishers 
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readability formulas thorite extent to control at least the grade place- 

■ * " * t • 

ipent* of passages t if notjthe revision or the Initial writing of the 

" " ^ • ~~ ~ • ~~ . 

p^asTages- themselves. Thus ,^Qrmu fas ^n^eeome se If «f0 If Winger 

prophecies in continuing to be applied to passages that are written 

' - * ~~ — 

to conform to them. . "» ' * * - ' 

The other criterion against which formulas are validated is an 

empirical measure of the comprehensibi 1 i ty of J>a9sages.* The most 

' - ^ v ^ 

promirtenCbf these are the McCall-Cf^bbs (1925a, J?25b f 1961) cpmpre- 

hension passages. Th&e ar* short paragraphs, mostly fictiorjjpr > 

expository nonfTctlpn; TolTowed~by "12 » 15" mulrljprle-chofce content 

questions. /Because o^he relationship between .the questions and the 

passages, the passages are feht to represent a set of materials for 

which there is some emplrjcal indication of wha\ is learned from them/ 

i » — f • 

These passages were administered to a large number of children of 
different grade levels- Two numbers were produced: *&e;^<k criterion 



of readability, which gives the ave/age grade level df^J^fdren who 



got i*t least- 5QS, of the que#S*fons correct for a particular passage, 
and C70, the- average grade level of "those who got 70% correct. Advantages 

of the^fcC^H"Crabbs passages are that ttfey were normed on a large 

' i » • * • • « 

population 1 , and there seenjs to be scfl^empt rlcai basis for grade pface- 
4re^t^ftf the passage*. There are se*erdl problems vfTth this ttest, - 

however. Are 'the flUestlqqs'toore difficult than the text Itself? 1 Afe 

• 0 ' ' ' \ * 
the questions^ accurate measure of learning from the text? Do the 
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questions measure something a child hap learned from the/ text or has 

-* 

brought to the text? Are there problems with testing e/ror fatigue, 

\ - * 

and~rtem error? Further, the original normlng did not /involve a wjde 

t \ / * 

range of readers, text types,, or tasks, and there is ytlll a^edagogical 

circularity in that the passages are similar Jrt many/ respects to those 

~ / ' 1 ' I ^ ■ 

from basal readers. And 'finally, problems arise Inr val (dating a modern 

* * / 

fo/mula agalns t^hlghly dated jnate rials/- and norms./ in one attempt to 
update the norms, however^ using fon temporary cm Idrpn reading the sarpe 
passages used in 1925 and answe<in<j the iqme questions, no significant 



differences were found except/af the I2tn-gi 
& Selden, 1978). . : 



(Jaeotjsoi), Kirk land, 



Cloze tests have been useiJ Independent measures of readability, 
too. The cloze proqedure- involves deleting every fifth: word from a * 
passage, wttjh the subjects' accuracy in supplying the exact mfssfng word 
indicating the readability. OnQ of the (Objections to it Is that synonyms 

*- r 

are not coupted as correct, although it is, claimed that this does not 

— . - , i 



t cowued as correct, although it 1 

affect its ^aitdi'ty. Another objection is that this test doesjiot , 

involve the satoe psychological processes as are Involved In feading a 

text and performing Some task afterwards; i.e., the cloze task Involves 



more 



looking back in the 'text for clues to the deletions ?o that Jt is mo 
. * tike solving a verbal puzzle than like readfng a text In a linear 

fashion. The correlations may be high because the verbaPproblem-sol vlng 
skills that are useful fn doing a clo?e procedure op a passage may be 

• " 28 \ ' 
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held by readers pho are proficient in various other 'kinds of comprehension 
■ * 
tasks. • 



Researchers^Jt HIE conducted a study - that serves to illuminate what 
is wrong with readability formulas and how they aVe used. The study 
involved the analysis? of the difficulty of the federal income tax ^ 
packagp in terms of the speculated ability of the average adult, to read 
. the package and use the information \n it. The General Accounting 
Office insisted some kind of readabi ! i ty^neasure being used to # verify 

that (a) the ordinal package was too difficult and (b) that, with the / 

^ * f * • 



reconroended changes, the resultant tax package' met certain levels. p The 
estimated difficulty was lowered according to the Dale-Chall,< tfarrls- 
Jacobson, Fry, and FORCAST formulas. There was a good deal' o^ variability 
# -among the formulas abt>ut the estimated level, as many as fpur' grade 
levels difference, -with the average grade level being about 12.5. Otber 
kinds of analyse*, such as detailed linguistic and visual format analysjs, 
were carried put to point out difficulties in the text without reliance 



on the formulas.- For example, there was extensive use of \4f t^branchlncj 
Structures in the forms, such as preposed Vf clauses, whlch psycho- 
linguistic research (e.g., Palmer, 197*) has shown to be diffjcu!t*to 

process, -and which Jar6 not accounted for io any formulas, except to 

* / * \ * 

tfce extent that sentence length reflects t^iis. If these structure* 

were moved to the other end ofxtfie sentence, the senferfce might be 

easier to process e\^r\ though it would be the same length. 
* 

• • r 

•• ' ■ 29.' . 
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It was estimated that oncrthird of the adult population confronted 
wi th the pr.Iqlnal tax forms wox\^ha^|^di ff iculty reading them, 'based on 



what little informafioa the;& Is on how, well adults react A set 6f 
revisions was proposed, 'moving conditional-' clause? to the end o$ 

4* ' . ^ t\ V. 

sentences, replacing some of the tecKnrxaj vocabulary, and doing a « 

— • * 

good deal o'f work with the visual format. , These revision lowered 

* • * » v . , < * 

the readabi lity level* by three to four levels (to about 8.5) nly 

by changing factors that are not di rect ly, measured by readability 

4 * , . * .<■ t x ' \ 

formulas. At the same fcjme-, .the IRS independently revised the tax 

_ ^ - ■ — r — - 



materials, .lowering the readability estimates by four grade levels 

(to £) mairjly by reducing sentence length and word difficulty, variables 

which are measured directly by the formulas. Nevertheless, when a 

reporter .tjave these IRS-revised forms to some bright eighth graders 

* ■ • 

in San ^pse, very few of tfietfi could fill put the form correctly, given 

m * •* 

simulated, data. (Unfortunately, there was no controlled field testing 
o f thri ^ re vis fed HJt E tax f o r ms on a. sample group of potential users*! . 
Jhe failure of eighth-graders to adequately comprehend text 

evaluated at, eighth grade Jevel points out what Is wrong wltif using 

* # 

readability formulas -for evaluating and controlling fhe difficulty of . 
a. variety of types p£ text. * One problem Is the publisher's use of the 

* * * 

formulas, not as Indicators of tfle generqj difficulty of a text, but 
as Indicators df what changes should be made tO} lower the difflctilty 

» * ' * r ■ 

• * s 

of ^particular texts** This not tfnly does not work, but it ajso Ignores 
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the explicit warnings of the formulas' Inventors^that_they are not valid 

~ ■ — * - \ , - • 

fpr-stich jnirposes. A-Jarge problem wi th the lHS*s attempt 1^ that the 

formulas used are based on the abilities of elementary school children 

and the properties of elementary "school texts, while the tax materials 

were written to correspond to an adult level (8-122^ There pre striking 

differences between the range of adult readers 1 abilities and that of 

learningxreaders in elementary school 1 . There are also radical 

differences between the type of tex^ used in tax instructions and in 

narretfve and ^xpo s r t^ry -prose, and there are strrktng differences — 
* * «* 

between the tasks of reading instructions and following them on a 

form, and of reading a paragraph and answering'quest Ions 1 about it. 

\ t ■ ■ 

Readability formulas are not sensitive to differences In ty{% 

.type of reader, or type of reading task. 

<* 

% An' Alternative ftodel 

V ___TlxLs Jbrings us to ao^alxecnative-raodel .fx>r-the_problem.of measuring _ 

/ . i . • ■> - 

readability developed by myself and Janice Reddish of the Anerican 

Institute for* Research. 1 The objectives behind developing this model 

are (a)^to proyide^ some bajJs for estlmating c and evaluating the reada- 

bllity of a piece of text, and (b) to come up with a technology that 

is self-enriching In ofder to Increase our understanding of what makes 

\ 1 
things readable. | This new model Is not a formula, but a more organic 

approach to the problem. It Involves first Identifying what the function 

or Intended purpose of the particular piece of text Is: I.e. /what task 
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* * * * 

is involved, who is to perform it r jaodwhat genre the text is. Those 
_three difrensions__ se em to be useful and necessar y wa y s to cla ssify £he 4 
function of a piece of text. It is necessary to have some seris^-of 
what kind ;pf . reader the piece is being presented to, since readers, 
differ, >flT5cj, the task is very important;, since 'different tasks 
(getting the gist of something, memorizing details, following instructions 
etc.) may Involve dlfferent;,types of Veading skills, Thli^d, materials - 
of different genre differ In their structure and*appl( cabi lity and 
can be used by different people to do different .things (see, Fig ure 1), 

( . 

Insert Figure 1 abou£ here. 
% ■ ■ — i ' * , 

0 * 

If one starts out, thtfn, tfy specifying the function of the text, /— 

* *» 
there is a body of scientific and practical literature one can draw 
■ * - * 

on to determine what characteristics the texts should have in order to 
.Tneejt, their functions. In the scientific lfi ferature, there .are 50 ^ears 
6T reWabTll ty research and TOO years jEf psycho tog fcffi research an hovr 
characteristics of texts interact with characteristics of people. 
-Jhere^is a practical body Of Hterature in the fnJeJds of technical 

writing, graphic design, and educational technology. To some extent, 

• *• * 

the design of a piece of text can be control led. experimehtal 1y; i.e,, 

o/ie can identify, control >y ^ffa keep track of what is being manipulated 
in the document* The next step is to) test the new piece of te^t through 
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a field. situation that tests the adequacy of the text against Its - 
o riginal function. Tha t eva luation js used to verify w hether or not 

the design precepts hold. , 

n ■ - 

This system does two things* first, It gives a basis for prag- 

matlcahy and realistically telling how difficult a document Is for 

its intended purpose, because an operational field test has been run. 

Second, this cycle of drawing on the research" literature and the 

practical guldel Ines F> systematical ly controlling the design of texts, 

and field testing the te^ts has Implications for enriching the validity 

of the scientific and practical guidelines. This Is <?ne alternative 

# 

to w readabl 1 Ity formulas. It does not account In detail for the way 
a given document Is going to interact with th^^ycho logical and 
personal characteristics of an individual user. WhatMt does do, 

* * 

is provide a technology for aystematlcal ly designing and controlling 
documents'. M 1 ' 



'Discussion 

" *" * 

Green : The A1*R alternative model to readability formulas kist 
— — — *„ • f* » 

described makes a great deal of sense, and'in fact? f Its *ln with some 

a « - 

research which I would like to. conduct. My idea was to copduct field 

f \ i 
tests of trade picture oooks which children In klndergartfin or first 

* « 

grade like to read, but which are rated higher (about third ^rade level) 

, - • ^. 

by standard readability formulas, to determine If such trade books 

really are that difficult*. A number of trade publishers have "Easy* 
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, Reader" series which are advertised as vocabulary-controlled and sometimes 

* -jzL ~~ 

controlled by readabil ity formulas aslwelU lypi caUy^such trade, , > 

1 ' * * / > . ' • • 

publications are much more interesting than basal readers. 

Another point brought up by Selden was that proponents of reada- 

biltty formula; suggest that they be used by teachers and assume that 

r . ' ' 

they will be used by teachers to fit materials to their students. 

* While I concur that teachers probably have not qop the ttme—lf In 

« " fact they have the Inclination — to conduct such work, t am opposed 

to such a suggestion on principle. The 4)ljXlQ$ophvLJbehtnd_that.^is^o- 

discourage teachers from relying on thei r instincts * but rather 9 to 

rely on the judgments of "scientists ,° whjch are labelled as valid 

and objective. This philosophy explains, at\ least ln\part, why 

teachers 1 manuals are often three times the length In pages and many 

more tunes the length In text- of the text being taught . 



Also "brought out by Selden *4 presentation was the comment that % 



applying Flesch's (JJ58) readiag case/human interest scale to technical 
docj*nents might *e?ul flnvblzarre texts. If the Art of Readable 
Writing (Fleseh, 1949) Is^tfy Indication, this is Indeed the case, 



since flesch practices what he preaches* The text Is very much like 

sppken language, and Is filled with direct commands and colloquialisms* 
* ♦ 

Selden; Oneway In which a tpwer readability score can be reached 
is by aycridlng semi-colons $nd replacing them with periods—which would 
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appear to be one of Flesch's strategies. NQtp-aJso that replacTngH 
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-technical .vocabulary with sjn^l£-words may forego accuracy, and may 
n$t account for jt he fact lhat the audience for which a te^ is written 
may be familiar with the tectfnl'cal words., This would be th£ case 
within the Government, for__example. 

1 Seidenberg : Several, issues have been brought up that are hardly 
debatable. One is that readability formulas have not been validated 
in a yali{J way/ A second is that people can abuse readability formal as 



*n various ways. For Instance", they might use them as a guide to 
writing texts, instead of as, a way of getting some additional informa- 
tion about something which has already been written. However, a 
lot of Issues remain open -to question. For example, improper t 
validation of the formulas does not necessaVily invalidate the entire 
enterprise of readability formulas.. There may stHI be a reason to 

X ■ / 

want formulas which will be easy to apply to text s^ and tTiejte-jntgnt 



^t±LI_be-a-theoretLca4-basis-on which to develop vaTTd Tormufas. The 

/ 

AIR type of alternative fs acceptable only If there is a farge amount 



of time and money available for the field testing and evaluation?. „ 

- • / - 

Certainly, such testing must be done for at least some texts some of 

V 

the time. I would expect publishers to oeact Instead by suggesting 

that^tfe* validation* 'tests be done properly on already existing formulas 

,* / 

or that, new formulas, be developed to perform the *same functions^* 



existing foi 



those already existing formulas 
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Selden: In my ,dl ssertatlon research (§elden, 1977}, I polnt'out 
fcl1 * * *p ■ i ' 

that the formulas fail to take Into account the syntax of the sentences 

at alt. Only sentence length has prevailed as a variable which i/idi- 

cates syntactic difficulty. Others had noted In the 1930's and \SkO % s 

that such features hs number of prepositions, number of Independent 

clauses,/ as well as other grammatical features seemedlo.be associated 

with readability. In thel970*s, a great deal of Chomsky an research* 



, was done to Investigate the effect of the number, of transformations 
implicit In the surface string on readability^ However, nobody had 
Investigated how frequency of occurrence relates to*how difficult the 
syntax Is for' the^feader. Hy hypothesis was that texts which contain 
relatively common surface strings would be more readable than texts 
which contain relatively uncommon surface strings. I went through 
andaborate procedure to measure this effect and to construct' a 



readability formula variable that would account ^For this. What \ 
found was that addition of such a variable (frequency 'of occurrence 
of surface structures) added significantly to the predictive power of 

the formula. The effect was much greater for older readers than for 

» v * » 

m 

the younger" readers. I needed a curvilinear transformation of the 

V 

^varlabl'e In order to get a significant effect, even for the older ' 

readers* The marginal change was Sxtrertely tow/ t was left with 
the pragmatic concl us I on that while this feat ur^ may be a .valid part 

1 Tk ♦ 

of^cfiBjdablHty, It might not be Important to account for H under all 
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conditions. .Certainly writing ski 11 and talent and pla^n good wrlt j^9 j 
would be at least as Important. / 

Adding variables to formulas Is an. extremely bulky process- With # 
the'advent of .computerized text analysis, the addition of new variables 
is becoming more feasible. 

The real puzzle concerns" the yaVJdatlon aspect of the problem. 
Why haven't additional vaUdatiojJjtudles f been performed on these 
formulas? How much is involved In takln/a different set of passages, 
applying the-formu|as , and comparing this to other criteria which 
indicate the difficulty of the passages? Virtually none of this kind 
of resea^h has been, conducted.- 

Seidenberg : Given the -seflous questions about the validation pro- 
. cedure, it Is difficult to Evaluate the meaning of studies which show* 
the effect of add ItlonaJ/ variables to be minimal. On psychol ingulsjic 
grounds, a variable such as frequency of occurrence might be expected 
to make conslderabl-fc difference. 

Selden*. Thls7ls In part a problem of regfesslon equations. In my 
study, I Would have preferred not to* have controlled sentence length,. 
One might then wonder whether or not sentence length was' Interacting 
with frequency of occurrence. Therefore,, it was necessary to control 
.sentence length. 'One effect washes 'out the 6ther. The answer to Such 
dilemmas Is experimental studies. Building passages which are controlled 
In all respects save one enable the researcher to see how comprehension ^ 
differs tjhen. manipulating the variable* 
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3* 'Readability Formulas and the Definition of 



the Task of Reading 
2 



/ 

0> 



Margaret Griffin 

In working with children, it becomes clear that the levels defined 
by readability formulas are misconceive^ fn that they are base^ on 



misconceptions, about the task of readjrtg. The ^formulas are least 



useful at the lowest and highest levels: At levels near the top, tne 
formulas characterize as readable what we would generally call poor* 
writing, and at the lowest level (Grade ,1 abd under) the Spache^ 
formula (1953) Is the only formula that can be used* T9 define 
•reading as progression through reading levels Is inadequate. 

In spite of these facts, readability research has an Impact on 
Instruction In several ways. First, teachers may accept the suggestion 
that* in the primary grades the goal of reading Instruction should 



to get children to read longer sentences with bigger words— 



a Strang^ goal for reading. Secondly, the formulas have an effect 
on the materials produced for reading instruction, sihce they are 
widely used by publishers and state textbook commi ttees^ 




\ 

The Inadequacy of defining reading as progression through reada- 
btlity levels can be Illustrated by looking at a stoCy composed by 
a kindergarten child, as reported'ln Vivian Gussln Paley's Wally's 
Stories (1981): •».•,. 



/ 
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.a little boy* "What language do you speak? 11 



•* ' Re&dabfllty Conference 

o ^ 32 V 

\/" * > ^ : ^ ; • 

A little boy lived afll alone Ln a de^p forest*. When he wanted**' 

\ # i * > 

to know a word, he.qsked Hons and tigers and wolves* They 

told hlfy pretend words, because he-oduldn't speak ♦animal 

language* One day he saw a lady and a man who didn't have 

„ „ do you 

) ''Animal pretend talk* 11 ) 

"That's tfK, g cause wefkan teac^ you people's, language* 

Which one do yog want to learn? 11 

"English." 
• « 

t^'Good, because that's our' language* What words do you 
want to* know? 11 * 

"Lion, tiger, and wolf." ' *' ' 

"You alrfead^ know them* You just said them* 19 
"Then anirnal ^pretend talk must be English* 11 

So they lived happily ever after* But the man and lady ' • 
kgew som^ words the boy didn't know, so they did have a lot ( p 
to teach him. m 1 ' 

The following is an example of a typical story to be .found in a 

basal reader for beginning readers: » * 

Bill said, "Look here, Ben., = p. , 

Look at the ducks." - !• * 

"Here, dudks," said Ben. 1 / ^ , 0 , 

"Look at this. , , 

\ 

Get it, ducks." # [ 

(Glnn 726 Level 3; see Clymer, Parr, Gates & Roblson, 1976, pp. 11-12) 
When Wafly's Story Is conpared with the kinds of stori<& .presented to 
beginners In basal readers, It becomes clear that the notions of ease 

defin6d by readability formulas do not correspond we 1 1 to children^ 

* * 
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actual abilities, Tlje child writer: Exhibited more awareness of 
connectives, for example, lb fact, the texts* written for children 
may actually confuse them as t6 Vhat reading Is for. * ' 

Goals of Reading Instruction ' 
"This brings up the major issue of the goals of .reading instruction. 
Vatnsle? (1981), argues that tbfr gdal* of reading. -Inst ruction are 

* * * 1 ' _ 

ideological by nature. There are three basic Ideological positions 

on the goals of reading instructions cultural reproduction, utilitarian 

-^^\ 

and interactive. ThoSe who see* the goals, of reading In terms of 

\ - * 

* cultural reproduction are concerned with the eventual dbl (tty ctf ,the 

t « ■ < 

reader to "read the work?, of the culture that are deemed inportant— 

Faulkner or Shakespeare, for example. The second viewpoint, the 

utilitarian orientation, is directed toward the ability of the reader 

« 

to read materials', such as tax forms, that are necessary from a 

* *. * * 

practipal point of view. The utilitarian goal Is a subvariety of the 

cultural ^reproduction 'goal ; in both views the reader is seen as an 

object, and strategies for teaching reading are likely to involve 

manipulation of the tekt. a. 

The thl cd, -interact! vfy/'vlewsees the reader as subject. There 

.are various Orientations wlthia this viewpoint, such as th^ romantic 

(exemplified, by bibiiotherapy) , the cognitive,**^ the social (as In 

the work of Kozol, 1^78^ and Frelre, 1970)- in the social -interactive 
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framfevforj*, the basic question to asfc In evaluating the reader/s ability 

. to reajja text Is; , Do pefcple alt Leyel Y get* the Judgment that they ^ 

v \^ fhav$ r/ad this text? ^ / * • . , I 

> 

Keader-as-Sub^cf vs. Reader-as-Qbj'ect foproaches 

^ -As an example of % thrf difference In approach be* ween* social ln|erfc 
f • active and reader-as -object models, consider a, text such as: 

{IV Through the years the nation Uapan has trained the 

*• • _ * * 

> ski lied workers needed'ln those factories. The result 

* * • • « 'i 

,1s a prosperous Industrial nation. 

• w 1 ■ 

* • y ' 4 

A typical criterion question that might be asked of a reader In a 

* /^reader-as -object model j,s: # ■ , * 

*• * (z) Is Japan of ten thought of as a well-to-do successful 

nation?* , ^"""^ * 



r 
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Frelre (1970) would argue that a question pf this type doesn't provide 
a very full criterion pf understanding. The relevant question, In a 

Social Interactive view, Is: What does It really mean to read a given 

# " • ■ » * 

text? Do you want people sTlnply to be abje to reproduce .Information , * 

of do you want;- them to have something like social Interaction with the 

* text? Hence, the social Interactive model would ask the examiner a 

* * • * ' / - * ' ' . ' 
. /question Jlke (3):-. * <* f 

4 (3l Is a question »Uke (?) a was Ore df whether the passage 
/ ' Is readaSlte to the answerer? • # . * * 

Readjpbl II ty, formulas,raay predtej: tl\e refidahl II ty"of a passage 

fpr many kids,, but Other factors are Involved, and It Isn't clear" 

, • • f f 

; ERJC . ' . * • • • : 
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that^rtttng a better readability formula wf If make any difference* 
What makes a passage easy to read I s not necessarily text- derived. 
Using a number of facers* It Is possible to make a text easier, but 
how do we knpw whether the text requires the same task after this has 
been done?* Why, for example, do we assume that'readlng the °duck M 
story Is the same task as^ reading Something like "Wally's Stofy?'^ How 
do we tchow that In making reading easier we haven't made It a different 
'task than what we want children to eventually be atrlQ to do? ^ 

v 

Cultural Attitude and Purpose - 

• « 

The ability of children to read a text Interacts with cultural 

* • . ~ 

1 values and .with the purposes for reading t\\e text* For example* In 

one first-grade group of children, the children were able to feed a 

- - • * t ' 

coiTplicated menu containing Idiosyncratic spellings and abbreviations 

in order to find out what was*)For luoch on Wednesday, JJbt could not 

read a "simple 11 text that said only "dodo bird" (under a picture of 

I - \~J * ' 

a dodo bird). In the tatter case, the children wanted to read the 

t 

t£xt as n doo-doo bird, 11 presumably because of the attractiveness of 
the taboo expression and the orthographic analogy wKh the word do, 
even though they knew phonics principles that should fiave given them 
the connect .pronunciation* In another example, an 8*year-otd 
confronted with' the sentence It was three long hours became so angi*y 
f that he refused to continue reading the teJtt on the grotinds that tKree 
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hours Is three hours. In cases such as these the "readability 11 of the 
text depends on factors of cultural attitude and purpose , and notion 
the length of words and sentences. 



, fodel of Differences 

The overall problem we are dealing with in trying to describe 

readability Is: How does a model of text features fit the material^ 

of texts? This should not be seen as a relative ranking question f 

for which It Is satisfactory If a model of differences caVi Jbe devised 

• • 

that fits some^ comparative ranking. Rather, we need to ask: Can we 
• « 

come up with a model of differences that fits all relative rankings 
of passages? 

Factors Hot Considered by the Formulas 

On the Flesch scale (1948), Graham Greene's The Heart of the Hatter 
receives a rating of 86, and Faulkner's The Sound and the Fury 97, 
Indicating that these texts are very easy. In general, the .readabi 1 i ty 
scales often give- strange and counterintuitive ratings. 

There do seen to be some ways of patching thing* up. For 'example, 

In looking at a text, the following factors could be carefully con- 

f * * 

side red In addition to the score on a readability .formula: (a) degree. 

of abstractness, (b) usage of metaphor, (c) unencoded inferences, fnd 

(d)* sophistication of topic* 
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Word Familiarity jnd ton text 

' ' v - - ♦ * 

There arc some sped ftp problems tha*t occur even with the more 

sophisticated kinds of formulas such as t-unlt analysis (Hunt, 196S) 
* • 

and Hotel and Gfcanowsky ,, s 0972} synrtactjc complexity formula. These 
■ 

Include the question of what Is meant by *V*qrd familiarity." The 
Idea that there is a single measure that ca n be~ cal led -1 V?o r<f 
famHtartty M t^ncfres the question^ of familiarity In oral vs. written 

* J A 

language, background of the redder, jand llje context In which ttje word 

ia used* Compare, fQr examp1e,«|^ai with (4b):* 
* ^ *r J> * " * 

(4a) Then he looked bafifc, and to, h*s great joy he perceived 

?' * * • 

"that, tlje Lillian (a bOatbKid gdne.*off In a wild 

d I' rect I on and . W$ yaw I n g > a 1 1 o$r%r the pond, 
* (4b) When a boat Is^fn'a race It must be careful not to 

In (jia) there Is^ sufficient 'tont^kt to enable the reader, to figure out 
the meaning of % tfje word yqw^ whfre*t*i (4b) It Is quite difficult. • 
^Readability formulas that influd> ^^asure of word familtaftty are 
not sensitive to ipch ailMief^nces, and wguld rate yaw In (4a) as Jus^ 
as difficult as^yS* In ,(4b). 

Context may also ena?>te-thfc reader '<flt<\ understand a difficult word 

via a self-defining structure, as in (5*1 : . 

' V 1 

(5) Beetles are one of the " few kinds of t Insects tibat jnakd 
# 

their own lights Many glow only whHe they are larvae, 
the young wormlfke form. * ( v 

♦ • 
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In (5)t the text defines larvae / but a readability formula would not 
take account of thl^fact. * j * 

SimllarlY, context may make words that should be "fami 1 for"* (and 
therefore easy) floj£ difficult, as 'in metaphorical usages* In a text 

• • • 

OS} . - . Jan had spoken a declaration of friendship that 
. would make other white men hate him. A particle of * 
white rock had detached Itself from that looming 
mountain of .white hate arid .had rolled down'the slope, 

stopping st t^l 1 at his feet, 

*» 

•the words rolled down the slope , for example, are made more difficult 



V 




by the facts that they are. used metaphorically. Again, measures of word 
familiarity discount this sjort of fact. 

v . 

Syntactic Complexity * % \ 

Measures of syntactic complexity are similarly inadequate for 

capturing the real level of difficulty of text for a reader. One 
» * 
"problem Is that- the number # of ways to classify* any given sentence is 

often extensive, there \% likely to be Httle agreement among classlHers. 

For exarpple, the distinction between headless relatives and embedded 

questions may b'e'vefy difficult to draw. y+ 

Freeman, larkin, Humphrey, and YSnofsky (1978) point out 

that; judgments of syntactic complexity must be made In terms of some 

» • < . 

theory of how language works, and cannot be determined simply on the 
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surf ace. structural configurations. An example 'is the dif-- 

« * 11 ■ 

" • * 1 \ 

ferdnces among' Joe 1 s* 1 ucky ' brother" , Joe*S lucky days an^ Htoe's lucky 
penny. The relationships among Joe, lucky , and the noun following 1 ucky 

are different In the thr'ee cases, but surface structure hidfes this 

t ' / 

underlying Complexity, rthile a syntactic complexity scale will treat 
**** 1 

th # e three as identical structures. Some other examples of this type 
of problem: Vte expect a good team to play is ambiguous, but possible 
differences I n complexity between the readings will b^ ignored by 
the formulas. A similar situation is the ambiguity rn some cases 

.ftf >pahgrirted 'qia^Hnns and hea dless rftl atjypg, as f n Hat te_ WO ndere,d_ 

■ . 

what Sally did, where -there may be real differences in reading 

difficulty between the two readings, "Marie wondered what it was 

that Sally did 11 and "Marie wondered the 'same thing that Sally did.' 1 

In some cases measures of syntactic complexity seem to over- 

predict difficulty, as in (7a): 1 v ■* , 

(7a) The policeman who is usually on -the corner wasn't there * 

• ■* 
today. , * 

(7b) The policeman usually on the corner wasn't there today* 

Note that (7a) contains a full second clause, -whereas (7b) contains ' 

a reduced relative clause (vlaJ/Kiz fleleti&n). The Syntactic Complexity 

Formula (SCF) discussed in Granowsky'and Botel (1974) assigns a 

complexity count of two to "dependent clauses" but only one to adjectives 

and prepositional phrases. Thus the SCF judges (7b) to be easier than 
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<7a). Based .on the work of Fodor and Garrett (1967) ar/d others, it 

jl seems likely that (7a) Is actually easier to process , k I nee It provides 

~* — -+^~ . „ ^_ _ § § 

t * mi 

more cities to the tJrtdeHytng structure , making gr^mmaxlxari re+at+ons * 

/\ / 

more explicit. $6 It appears that such a measure of s^tacfrn; complexity 

/ 

as the SCF, as welTas one based on sentence- length, will make the wrong*/ 

prediction here* — 

A somewhat similar situation occurs in regard to passives, which 
will automatically be rated by formulas as more difficult than acttVSs. 
Vffille it may be true that passives are harder in general, actives are 
cleai^y more difficult In contexts which* topical I zq| the underlying 
subject* Thls ( accounts for the strangeness of (8b) as a TV announce- 
ment: 

(8a) Tarzan .wiM not be seen today because of the following 



presentation* 
■ 

You wi 1 1 not ! 
presentation* 



(8b) You will not see Tarzan today because of the following 



* (Freeman et al . , f978) 

« 

Clearly, the passive Is better Because Tarzan Is In topic position, where- 

* * 

« • ■ 

as (8b) seems to be stressing the viewer. Formulas which automatically 
consider passives to be harder to understand than actives, regardless 
jof context, will often make wrong predict!j>ns* 

, There Is also a problem wi th /'sirrple sentences, 11 as Illgstrated 

♦ 

by the contrast between (8a) and (8b): 
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(8^) 1 like to telephone. 

j 1 want to telephone Sam. , * 

I dial his nunber. 
r % Sam's^elephone rfngs. ^~ 
I hear a noise In my telephone. 
. - *7he noise tefls-me Sam's telephone Is ringing. 

(8b) I like to telephone- I want to telephone San>, so I 

dial his nunber. Then a nolse Tn my telephone tells — * m 
me that Sam's telephone Is ringing. ^ ' 

As ppinted out by Freeman et al. (1978), the use of separate sentences 

suggests that each fact Is Important In ancflbf itself and leaves the 

feTaTTbn shjps among theM actsnSTspecTTTeJT He f e^l)f^c^Tex7"ty ,~~ ~~ 

however, would rate (8a) as much simpler than (8b). . * 

Level Inq-Out 

^ Overall, readability formulas do not seem to be adequate as measures 
of difficulty. On a more general level, It may be that the notion of 
"level" /such as "Jevel" specified by a formula) Is not useful In the 
teaching of reading. Hor does the use of the concept "level" seem to 
be very useful when applied to grouping children according to ablHty . 
for th^-purpose of reading Instruction; the result of such groupJng Is 
that the* gap between -lower-group arid higher-group chl Idren widens. 1 It 
'may be that "level Ing-oiit," which doesn't work In (grouping children, 
really can't be expected to wojk^ln evaluating texts either. 

This View Is supported by the wprk of He'ss and TakanlshJ (197*0, 
who*have found that properties of classroom materials do not have much 
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IjnpAct on the amount of tftne spent on text, a measure whlch_Js correlated 
filth comprehension of tfie text* Rather, Interaction with the teacher____ 

seems to be the* Important factor. * - 

* 

Research suggests that the reading level of a text has very 

■ • 

little, to do with how well a child c*ads 4 or hot-quickly the child 

■ ■ t J 

learns to read. This Is indirect contradiction to the learnlng-to- 

- _ _'_ x * 

read theory which suggests a hierarchy of easiness to hardness on the 

basis of texts. The questions which must be addressed ff we are to 

develop a theory of the reading task are: How important are texts7 

Are characteristics usedly the readability formulas correct? Are 

- » * • 

sentence length and word familiarity hierarchical features? Does the 

tafk of reading have to do with length and familtartty7 If we answer 

this last question In the affirmative^ how do we know that this 1%. an 

appropriate definition of the task of reading? Tjjere is'a confusing •' 

blend of tQxt-specl f 1 c and general izable characteristics In the learning- 

to-read theory, such as the assunptlon that the ability to read Here ti 

t n */ * 

a duck wl 1 1 generalize to the ability to read. He re lb a iamb, this 
not a very safe assumption. 

Zone -of -Proximal -Development Tutorial 
' « » 

An alternative way of studying the process of learning to read I* 

* . 
the zone -of -proximal -development tutorial* The gogf of thl* type of _ 

study, as exemplified in the work of WoTf (1976^ HcKenzte (1977), and 

• , , • » 

Etter (Note k\ t Is to answer the questions: What goes on in interactions 

"T *• 
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„ between text and novice vs accomplished readers^ The answer tOyth.U. 
question should reveal the components of the heading task, and- should 

■ * 

demonstrate what Is relatively accessible to the novice, which Is 

* 

the real criterion of easiness. * 

* • « 

The value of taking this point of view Is demonstrated by studies 
such as Ferelrro's (1978) work with prereaders, In which She used 
Placet! an methodologies to determine what pr^readprs think Is Important 
In reading. One of her findings Is that 5-year-olds do not believe 
that verbs .are written prt the page. Lurla (1977-78) • also dealing 
with the question of what children expect wten they go to the literacy 
task, has stressed* the Impfortance of .children's idiosyncratic writing 
and of discovering what the child thinks Is writing. Tfils approach 
takes as Its starting point the question: What do children already 
know? * * ■ 

' Discussion 

Char row: Studies of the literacy demands on government clerical 
workers show that .many ^ noJfc-toon*M -educated people lalm not , to be| 

able to read, but that this Is because reading Jias been defined In a 

• » * v ■ 

• * «■ * * 

certain way, and In fac^ in their work these pebple perform a wh^Ie 

range of tasks that one cogld recall '^reading," Jnclud.fncj reading whole 
pieces of text. 

* Griffin ? The zone-of«proxlmal- development approach would Involve 

' % — ; * r - 

'sitting down with one pf these people and figuring out what can be done 
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to make them able to do more than they're already doing. Then, when % 
this has been done with a number of people, yoij, can begin tp abstract 
out of your findings a scheme of what you can do with people at dlf- ^ 
ferent "stages." Wiat you enc) up with Is not b s6t of levels related 
by quantitative differences; Instead you tend to get a set of 
functionally different systems. 

Green: One potential device for showlrjg the relatedness between 
sentences when connectives have been deleted Is paragraphing, bat In 
prttnary texts there Is a tendency to eltmtnate this sort of structure 
'as well. 

Griffin : Paragraphing is probably not, very "aval lable *tq the teacher 



as a device for showing connectedness because bf the tradition of oral 
reading In Which the text tends to be brc^en TrV strange places*by 
shift c, .rn from child to child, perhaps on a sentence-by-sentence 
basis. * n 

Pav I son ; Although It seems' clear that there may bp difficulties 
Involved In drawing Inferences In eases where there ts no explicit 
connection between sentences, It Is difficult to think up an experimental 
paradigm Tor. studytng-'jdiat goes on during reading when this Is the case. 

Grlf f.ln : There art some research programs that purport to study 

the differences between texts wlttvand without connectives* but .a .major 

" * - 

problem ts thpt you don't really have the same text when the connectives 
arc In that you do when they are taken out. Apparently sometimes having" 
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only paratacttcs« Is harder, and sometimes having only syntactics ts 
harder, and sometimes changing from parat^ctlc to syntactic changes 
'the, meaning of the-te\ct* 

Osborn : Given the state of our knowledge,. It *s very Important that 
all steps toward Improving reading Instruction be taken with a heavy' 
dose of tryout in classrooms. Fok exarrple, even though (5) contains 
..aTdefjning structure, children may not Ibe acquainted with thfe coh~ 
structlon and may not realize that larvae Is being defined. 

Griffin : This Is the sort of thing that wquld be found out In a 
tutorial. 

, Osborn : But we cannot count on teachers being sensitive to it 

In the kinds of classroom settings In which most reading Instruction 

Is now being carried out. . • . 

Griffin : There are ways of basing an actual curriculum on the zone- 
* ■ . 

of-prox! ma I -development tutorial , such as. that described Irf A. -K. 
■ Karkovi's The Teaching and Mastery of Language (1379)* 

4. Jobs You Shouldn't COunt'on Readability Formulas To Do 
& Bertram Bruce and Andee Rubin 

It Js easy to caricature readability formulas— to find a case 
where a readability formula rating Is errplrlcally Invalid arid counter- 
Intuitive. Without appropriate analysis, however, such an example does 

m ■ 

not show why the readability formula falls nor does* tt distinguish among 
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* * 

different situations in which the formula might be more or Jess appro-- 

priate* In this paper we look at some cases where the formulas seem 

to go awry and put" them into, a more general framework based on questions 

like tfyf following: What are the factors that affect readability? 

Why does one need a measure of readability? What are~the assumptions 
■ 

underlying readability formulas? " — 

r 

In considering the factors that affect readability, let us fifst 
take' a step back and, without ?py preconceptions, think about the 
kinds of questions a reader might ask her or himself in determining 
whether a particular text is difficult to read. Questions, such as 
the following might be relevant: * ' 

1. How do I feel?* Arp I tired? Huggry? *Do my eyes hurt? 
Ata I distracted. or preoccupied? , fr 

2. How interested am I In this 'topic or this story? ( 

3. What do I already know about the subject? Do \ have r 
enough background knowledge? 

k. Hew similar is the writer's language to mine? 

5. How plausible to me are the writer's presuppositions? 
What do I have to take for granted in order to understand y 
this text? " Which of^ttiy own beliefs must I temporarily lay 
aside? . * 

6. Why am Treading this? -Do 1 want a clear ippflei of all 
the facts presented in the^artlcte or general 
understanding sufficient? Is my purpose mereJy escape? * 
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7. How long do [ have to read "thls?^ How does this limitation 

* . affect my^ reading goals 1 ? * Z "» , 

8. : Whqt do I want to do wtth the Inforniatlon l/get? v ' 

, this ij>ere beginning of a list reflects tlhe personal, Interactive 

• * - y . N - «• t ' - 

nature of reading. Notice that each question examines the relationship 

St 

of the -text and the reader; none relates to the text jn isolation. In 
this view of reading, tt ts bizarre to think of a text as having a degree 



* of readability 1n and'df Itself, a*part from considerations df the 



readef and "the* reader's purpose* ' 

? 

In spite of. the plausibility and Importance of this image of reading 

and readability, readability formulas which purport to evaluate texts*- 

out of the context of the reader-text Interaction have continued to 

flourish. One obvious ~eason fort their growth Ts that a number of jobs 

e^Ist for which a simple measure of text difficulty would be enormously 

^useful. These tasks include : designing (writing, selecting, and 
* * ■ * 

.. „ adapting} texts appropriate to the level of a child in school, choosing , 

among trade books for children^ choosing passages for tests, evaluating 

1 s ' 

difficulties In reading; making writing clear for adu)ts t designing 

materials for, spectal^populattons; and writing and evaluating materials 

* * 

• to be used In research* We want to consider hefr a few of the jobs for 

which readability measures are or might be used, specific: examples of 

each, and problems that Come eg In using the formi^las In' these ways. 
^ I * * 

\ ' , > - * * * 

t(fn some cases, readability formulas ar&^not used explicitly, but.slmilar 

considerations of vocabulary and sentence length are applied less formal] 
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For* each problem wp Identify we would llke'to ask: Is the problem In • 
—-this example an idiosyncrasy , or does It revea| some flawed assumptions 
underlying tAe' readabll fty* fffeasure? After looking at several examples, 
* wewllj^be. In 3 be tter posi tion tp address wlder^ Issues, such asj Are ^ 



c * 



measures of readabl 1 1 ty useful for some Jobs but not 6thers? Are there 
any jobs for which such treasures* are useful and appropriate? 

3ob 1; TV Captioning foi* the Deaf 

' • ■ ' $ ' j * - 

^ i Deaf adu1ts*In this culture are doubly .Isolated from mainstream 

soclety—nojt only ls.lt difficult 'for themr to ccSmmunlcdte with most other* 
- * ■ ^ i. ^-" r ^ 

people a found thtJ* t\ut" they aTso tack access to the tejevtston programs 

so many people watctv^jWiG process, of captioning TV shows must take Into 

* * -4 ) " * * 

P account tw» characteristics of deaf aduUs 1 language capabilities for 

4 * /^rfilcti readability formulas could plauHbly provide some assistance. 

▼ T first* deaf adults' reading rates ^re generally lower^han those of 

hearing, adults and significantly lower thafKthe rate of oral presentation, 

S> so Jthat a simple transcription of the audlcrportlon of wshow would pot 

r 

be ncadable In the tlflfc available. Second, deaf adults are believed to 

r 

have more limited syntactic and vocabulary abilities than hearing adults. 
- s = - ■ — ■ > ^ T 

Since readability formulas focus 9n these aspefcts of text^they are 
obvldus cand4da|e$ TBr evaluating captions. The following Illustrates 
^ what fiappens wheh captioning Is done with prlm^corts^deratlon given to 
* • word choice and Sentence construction. 
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• Exarr) P le * The following texts ^are two versions ofj Introductory * 

* • * 

torments to a television documentary on Arab-Israeli relations. The show 

Starts with a picture of Marilyn Berger speaking on location In the 

l m 

Middle East \ Throughout the passage, we see the same scene /so virtually 

no Information IS communicated In the video portion of the show. * The 

first ( verslon below is the captioned text; the second is the original. 

(1) [1] I'm Marilyn Berger.' [2] I first came to the Middle' 
I East 11 years ago after the 6-day war. [3] toany Israelis 
thought lt*would be the last war. [A] This program Is not 

• about armies or difcj^nats. . [5] It \\ about 2 f ami l ies caught 

In the Middle East conflict* [6]' The program was filmed 3 

, . * 

years ago. [7] But the sarpe feelings remain today a£ they 

haveTor the last 30 years. [8].The feejlngs remajti although * 

there was hope from the historic visit to Israel by Egypt's 

President Anwar Sadat. [9] This program was the Idea of an 

* Israeli TV producer and a Ha/vard professor. [10*] They worked 

with an Egyptian newsman. [ll]*There Is no attempt to decide. 

• who is i^i ght o# wrong. [12] The purpose is to understand the 
conflict from the point of view of the people who have the most 

^to win or lose. fl3] Recently there ( have been more deaths 
and violent statements. ,[l4] In the fo 1 lowing prog ram we will 
'hear the softer voices which aren't often heard, but which are, 
no less typportant. [15] One family was filmed In Egypt. 
[16] The other family wa* filmed In Israel. [17] Both families ' f 
have had a great ]gtfr~the cffath of a child. [l8] The fllrn^ 
- team was surprised and pleased *by the sympathy that e?ch 
family had for the other. 

■ \ 
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W I'mjHjfrllyn Berger. I first came to the Middle East II ' 
years ago, just after the 6-day war—the war' so many Israelis 
thought was the one, to end all wars/ What you are ^bout to 
see ts the fjrst in a series about the Middle East. It is not v 
about armies, or gpy e rnrgen ts -or jdipj omats . \j^_s aboutLPeople 
and families caught In the conflict. TJie program was filmed 
In Egypt t l(i Syria,, In Jordan andaebanpn, and here in Israel 
some 3 years ago. But the same feel ings^^xhe^same ^dl Jerrmas 
persist today, as they did 3 years ago—indeed as^they haye 
for the" last 30 years. They persist despite Egyptian President 
Anwar Sadat's historic visit to Israel Jast fafr, and all the 
hopes that vis ft aroused here and around the world. Before we 
begin this series, a word about how it all came to be, and 
about some of the unusual people who were involved In Jt. 
Back in 1973, Zvi Oor.-Ner, an Israeli television p/oducer, 
and Professor Roger Fisher, of Harvard, wh6 shared a concern 
for the Middle Earst # and an interest lo television had an idea. 
By July 19|jt, the Idqp had t^ken'shape in a series of broad- 
casts. Fisher and Dgr-Ner became part of a team that Included 
Professor Nadav Safran of Harvard, a leading Middle East 
scholar, Mohammed Salmawy, -an" Egypt 1 art newspaper man, and a 
group from WGBH In Boston. Each program in their seHe£ is 
devoted to a specific aspect *of the Middle East conflict. It 

does* not try to resolve who Is right or wrong, If Jndeed there 

„ • " * 

is a right and a wrong. The puFpose was then, and Is now, to 

try to see the conflict through .the eyes of tlj$ people who have 

most at stake; to seeMt as they see Ijt* Recently there have *« 

been new casualties, more deaths"Soldiers and civilians. There 

4 

have been foore of the kind of violent and extreme statements 

that are so much a part of the dialogue .In this paVt of the world* 

In these- programs you will hear something dlf ferent-^ose $pfter 
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voices that may be drowned ,oiH so easily, but which are no jess 

* * * * 

-im^jrtant to UsterTto. The first broadcast in the series that 

fou'wlll see tonight centers on two bereaved B f ami! I es —one , _ 

' Egyptian, one Israeli — following the greatest loss a parent can 

AJ5_u£feil^the loss of a dilld. When Zvl Dor Ner, filming in 

Israel, and Mohammed Salmawy, filming In Egypt, compared their 

separate efforts, both were surprise^ and pleaded,. The sympathy 

each family had for the .0 then was something neither map had 

* 3 
expected. 

• * 

The readabi 1 ity' level of the original text is ll.ft, that of tj 
adapted text 7.k on the Fbg readability scale (Gunning, 1964). 

Problems- The problems with this adaptation arise mainly because 
the higher-level discourse structure of the t6xt was neglected fn Its 
construction'. • ^Fhus, although Individual sentences are "simplified/' 
the overall text Is less coherent* For example, the deletion of the 
repeated phrase "the war" from the original text causes a shhft in 
focus in the adapted text (between sentences [2] and [3]) to the Israeli 
beliefs. The result [f.a garden path readihg of sentence "[3} which 
at first suggests "Many Israelis thoOght It would be, Marilyn Berger's 

l&$t visit," In the original, the focus is maintained. \ ^ 

* ** » 

A simj lar problem fs the rough transition between Anwar Sadat 
(sentence [8]} and This program (sentence [9]). Organizational material 
from the original ("Before we begin *he series, a word \ . .") has been 
deleted, leaving \\ to the reader to .forge some connection between Anwar 
Sadat and the progrW-*-' 
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Third, Iir the adapted version It ts unclear who Is nqt attempting " 
to decide in There is no attempt to deci'de who Ts right or wrong 
(?enjtence [11]). Is it the producer, th^ professor, the newsman, or 
Anwar Sadat? The original contained sufficient context to make the 
answer clear. * ' " * 

Finally, the poor connection between the last^two sentences ([17] 
and [18]) tends to lead the reader to the unfortunate initial impression 
.tha.t the film team was surprised and pleased by the death of the'chi Idren. 
In t/iis case the origfnal text also has the potential of being misunder-, 
stood, but .the adaptation has made Jthe -problem worse by deleting the 
mention of "efforts." % 

In general, the problems with th is^ selection appear to follow partly 
from the. rate constraints imposed by captioners. Captions are written 
in general for a reading rate of 120 words per minute-regardless of 
^the content pf the caption or the video. Since in this particular ^ 
-case the screen showed only a talking head, one approach might Wave 

* # s 

been to increase the length of captions (and therefore the presentation 
.rate)' so that the text .would hdVe been more coherent. - ~ 

Assumptions* One assumption abput the use of readability formulas 

> ' « ' ~>" * 

relevant to this job is that they are baling applied to "honestly written" „ 

material, I.e., material that has not been specifically Written down 1 * 



y to meet the demands of J^he readability measure. One reason that measures 
of vocabulary and syntactic complexity work at all is the strong* correlation 
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between *real conceptual complexity and complex words ?nd long Sentences- 
In trying to adapt material to Ht a readability measure, words and 
sentences may be changed, But conceptual compJexity remains ^constant » 
Thus, a formula ,be comes ah Inaccurate reflection of the passage^ 
difficulty. 

^ toother assunptJon of the adaptation Is that higher-level strutitfrt* 

• A * 

can be Ignored* What is most often deleted is organizing material and 
connectives, resulting 3 In the kinds of *prob'le<ns. we saw, abova, * , , ft t 

* . 

Job 2; Elementary-School Trade Books 4 

•* 

Trade books written for elementary school children are often graded 
via readability formulas so that they can be matctied by teachers, 
1 ibrarlans, and parents with the readlqg abilities of children. Even 

1 • Cm 

* 

thojjgti it mtqht be argued that young children have very Individual 
tastes, these readability ratings are made on a 'uniform basis, focusing 
on vocabulary and .sentence length. Some readability formula advocates 
might even argue that word-level considerations are of primary Importance 
far young children whose reading vocabulary Is sharply limited. As 
just one example of the pitfalls of this approach,* let us look at one 

1.1 *V 

trade book which , according to readability formulas, should be easy 

* M 

to read* *V 

Example, Don't Forget the Bacon (Hutchins, 1976) Is an elementary 
— — , 

school trade boo^ which scores' between grades J[ and 2 on the Fry (1968) 
scale. The took concerns a Uttle boy whose mother Is sending/him on a 

. GO 
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shopping errand. Onnhe first page the reader Is shown the mother holding 

a basket and colrf purse and saying to the little boy {via a cartoon-type 

balloon), "Six farm eggs, a 'cake for tea, a pound of pears, and don't * / 

""forget the bacon.*' 1 ^ On* the next page the little boy is pictured carrying 

the basket and coin purse, walking along a $treet where there are three 

fat people standing pn the corner, and thinking (via a cloud figure)* 

* „ * 

i'Sl x fat legs, a cake for tea, a p.ound of pea>s, and don't forget the 

.bacon. 11 The book proceeds In this fashion, with the text consisting 

only of the little boy's thoughts and a few similarly structured ipstances 

* 

of speaking to the storekeeper and to his mother* 

Problems, The general impression most adults get upon reading this 
book is that it i s - VoJ difficult for early elementary school children. 
In addition, we have Interviewed several third and fourth graders who 
fouid the book (but not the Individual words^dlff icult. One reason 
for ^he discrepancy, Is that much of the information in the book is 

communicated In the pictures. Even the fact, for Instance, that the 

" * « 

little boy Is walking down the Street Is not explicitly stated, and the 

\ 

various scenes wh ich change his memory of what to bu y are pictured, not 

• * * 

described. 

Second, enjoying the book relies heavily on appreciating its 

symmetrical structure. The little boy forgets the i{ems on his list 

one by ofve^Sy going through a series of transformations of the linguistic 

» 

material he Is trying to- remember, then remembers the items in the 
opposite order, unfll he arrives back at the original list. 
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A third aspect % of the book not captured by readability measures 
fs the important distinction between thought and speech, indicated by 
the standard bat toons and ctouds used In cartoons* 

Finally, even though the book would not be opnslderbd poetry, an 
appreciation of rhyme and how people us* It as a memory aid is an *i 
Important "component of comprehending this text.* thus, the potential „ ^ 
sources of difficulty for the book are^not attributable to Its word 
difficulty or sentence complexity, yet these are the only factofS that 
readability formulas consider* % • ** „ m . 

Assumptions. The problems discussed above Illustrate the invalidity 
of the assumption that it Is sufficient to evaluate the readability of 
a text in terms of lexical and syntactic factors alone, Ignoring such 
factors as pictures, graphic conventions, and rhyme. 

Another assumption often made In th& use of readability formulas 

Is that It \% possible to apply formulas which represent statistically 

- • - * 
derived 'averages to individual books and to. children* In the case of 

Don't Forget the Bacoff, this assumptfofi Is clearly unwarranted, as this 

book seems generally too-dl£f leu U for tt s s u pposed age range. Such . 

examples will always come up in using any statist teal ly-derl.ved quantity 

a$ a predictor |n an individual case; this fact urges caution in this ' 

C9mrrpn use of readability -formulas* 

. Job 3: Reading Comprehension Tests * 

Readlag comprehension tests are another domain In which written 

materia) must be graded or scaled. Although we are n$t sure what part 
■ « 

62 
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readability formulas play in the construction, of reading .te&ts, we sus- 

pect that either the formulas themselves or related considerations of 

, vocabulary and sentence difficulty come Into playj In th£ screening, 

selection, ordering, and normlng process* The f6I lowing^ example also 

/ 

illustrates some assumptions abtfut the propels of reading and compre- 
hending texts which aft^ common to bpth reading tefsts and* readabl Ilty 
formulas* x 

Example * The* fol lowing Is a passage A "frdm the ETS Cooperative 

English Test of .Reading -Comprehension, I960: " 

As .to clever people's hating each other, I think a 

# 

little extra talent does sometimes make people jealous* They 
f become Irritated by perpetual attempts and failures, and it 
hurts their tempers *apd disposition^ Unpretending medlocri ty 
Is good, and genius Is glorious; J>ut a weak flavor of_genIus 
in an essentially common person Is detectable. It spoils the 
grand neutrality of a commonplace character, as the rinsings 
of an unwashed^ wineglass spoil a draught of fair water. No 
^ wonder the poor fellow who belongs to this class of slightly 

favored mediocrities* is puzzled and vexed by the strange sight — — 
of a dotm-metr of high capacity working and playing together 



l/i harmony, (p. 7) * * . 

One of the comprehension questions asked about this passage Is the 
following: 

The writer suggests that persons of exceptionally great ability 

(aV t;end to Uke and appreciate one another* ~ 
\ * 

(b) dislike the company of ordinary mefl. 
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(c) are likely to be jealous of one another/ 

> A * 

? id) are .essentially common except for their genius. 

Problems* Although the writers of the' test consider (a) to be 
the correct answer, some people choose (and can justify) (c) as their 
answer** People who select (c) InTerpret thfe first sentence to mean 

that' cISver people are jealous of one another (the same Interpretation 

■ ■ 

those who choose (a) matyk), but then equate "clever people" with 

uses.' 1 Although there are other Indications in the passage that 
the author does not consider clever people" to be geniuses, this evidence 
does not chang? their oplnTon, so they assert that "persons of exceptional 

great^abi 1 ity are likely to be jealous of one another." In our admittedly 

j 

f imi ted. sample, those who chose j[c) considered themselves members of the 
* * 

lowest of the three groups of people identified by the author (ordinary 
people, slightly gifted people, and geniuses); one* is terrpted to 
conclude that they were therefore less conscious of the Intended , 
differences between getiiiSse$ and clever people. Ev£n If this . In'terpreta- 

tion is Incorrect, It points out how quite personal differences (In 

- _ __ _ _ 

thjs c*$e t In self-image) might affect comprehension. ^ 

Assumptions. Users of readability formulas mtist assume ^ftat the 
Intended reader and the ratfcr of a passage do not, differ on whatever 



measures (vocabulary* life experience, purpose for reading) are relevant 

* m 

to Jhe eyaluatton of the text. In the case of a reading comprehension 
test, the "rater" -may be a person, a committee, or a normljjg process; ^Ith 
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-readability formulas, rater characteristics are derived from the popula- 
tlop on which ^exts are graded, £nd^are reflected, for example, In the 
cfypice of words, ^or vocabulary lists. Where differences^ exTst (as 
they do most obviously here and In a later example), the use^of 
readability formulas (or standard! zed comprehension t^ests) is less 
justified. 

Thfl passage Illustrates for a second time the Importance of honestl 

written material. Since this nyterial was obviously not written to be ; 

excerpted, but was intended as part of a larger text, this method of 

'constructing passages for tests violates the assumption of honestly 

written material , 

. Finally, it seems clear that the assumption that language ratheV 

than conceptual con ten j: is the main determiner of text difficulty would 

hot apply to this passage; the conceptual content is quite complex and 

could probably not be made any easier (in fact, might be made more 

* * 
difficult) through such procedures $s shortening sentences and 

simpl I fyiiffj syntax* m ~ * 

< "» 

Job *i: Remedial .Reading Texts 

Junior-high and high-school age students who have trouble'in reading 
present a difficult problem; Their reading skills may be inadequate 



for the texts directed at their age level, while texts which they could 
read easily are typically directed toward much younger, children and may 
be boring or embarrassing for them to read. One solution to this- problem 

65. 
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is to Identify a sejt of texts" that. score low on readability formulas but 
* ■ » * 

have high Interest value for the older 'student. 

4 

m f # 

There are several criticisms, to be made <Jf ,the hlgh-lnterest/low- 
readabi 1 i ty-score solution. First, it i$ extraordinarily difficult to 
select high interest texts for an Individual student, especially one . ' 

c 

j* ** 

vho is not 'much of a reader. Giving the student the power to choose 

(and reject) texts might be more effective and just fas pedagogical ly 

sound,/ Second, ft might be more useful to search for a meaningful 

text and task for which the student wotjd be ytlling to Invest some 

effort* ^Third, a low readability score is no guarantee of true ease 

of reading. This is particularly the case for texts constructed or 

adapted to fit a readability formula. . • * 

Nevertheless, readability formulas or their kin appear to be 

widely'used In* designing and, choosing remedial reading fexts.. The 

two examples dlscusse/cJ below illustrate some of the factors this 

approach fails to cope with. Both .passages spore about grade.? on the 

Fog readability formula. 

•* Examgje] I ■ The first e^prtffle* "Indian Occupation/ 1 is from 

Clue Magazine, No, ^ (education .Progress Corporation, 1972, 1979). 

ldlans^iad not heard from the government. The . 
it for Alcatraz was still not settled. The Lndians were * 
discouraged and angry. They did riot know if their goal could 
be reached. Some people wanted to tear down the buildings* 
lf The Vftljte Han Is our foe, 11 they said. "He took our land 300 
years ago.** It's true! The White Han wrote treaties, but^they 
were all a hoax, 11 * - 
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• 'Other Indians said, ^Walt! We must build a place here % 
that We can boast about. We must h^ve a school. ItVs dangerous * 

for our children to roam through these old buildings* We ntfed 

* * ■ 

'food, too. We must hoe the soil and pliant ^tomatoes f potatoes , 

and fruit-" \ ^ ^ 

Suddenly someone roared, "Fire! Firei'i A fire had started 

in an <rid building* Unfortunately , the boards made gootf fuel. 

The flames soa'red'hidhl* there was no water to m so$k the buildings. 

The only wateY on Aicatraz was the drinking water brought by 

the boats. The Indians had no pumps to bring water out, of the 

bay, Finally, tffe roaring fire was reduced to coals and burned 

itself out. There were no clues to tell how It started* 

' - Other problems came up. Food and wafer did % not come when 

they were du/. Boats cruised by, £ut they didn't stop at the 

Island, So/ne of the Indians, began to Icfaf* They forgot about" . 

the! r oatf/to work together. Richard Oakes, decided to leave 

the Island. Others said, "If he "goes', we'll* go too*" Nobody 

could coax them to stay. # * 

Problens / There are numerous problems with the passage, most of 

which should be obvlousr on a careful reading. To mention just one; 

Note the" bizarre intrdpucfton of Richard Oakes # at^j;he end of the story; 

he turns out to be the central character, but has never been mentioned 

previously. • — . * 

Part of the difficulty with this text probably results from the 

■ ■ * 

attempt to maintain a particular readability level, and. part from the 

effort to Introduce the written f.orms of particular sounds (longhand. 
« « * ■ 

u). Maintaining a coherent story line -clearly was of secondary Importance. 
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, Assumptions* One assumption this use,of readability formulas 
Illustrates Is'jthat one can afford to Ignore higher-level structure In 
assessing the readability of a text* This Assumption Is, of course, 
violated, by *texts which have no real coherence, such as the preceding 



c one* 



A second assumption Is that a text need not be honestly written* 

•I , - • 

This material was written; just to do the teaching function, and thus 

jdoes not qualify as •'■honest ly written* 11 „ 

• • * « i> 

Example 2, A second remedial reading passage Intended for older 



children is "Shine 11 from the Bridge Series (Houghton Mifflin, J977; \ 
see, Sijnpkins , Holt, t Slmpklns, 1977). ^The following is ati excerpt: 

Shine was a stoker on the Tl tanic . The Brother, he 
shpvel cdal into the ship furnace to make the engines go* . * 
Now dlg\ .Check what went down on the day the Tl tanlc sunk* 
Shine kept on goln9 tfp to the captalnr of the ship. He kept 
»d* telling tfie captain that the .ship was leaking. 

Shine run on up to the captain, and say, "Captain, 
Captain, I was down in the hole looking, for something to eat* 
And you know what? The water rose above my feet* 11 (pp* 1-4) * 

.J 

► Problems. While, the text from vrfilch this passage Is taken does 
s^em tfi be better as a story than tbe preceding text* It poses some 

problems for white students because It Guilds on oral language, in „ 

* *■ 

' ' • ■» 

particular on Black English Vernacular (BEV)* Some misinterpretations 

whl te students made when reading passages from the Bridge series were 

(a) to read brother as meaning male sibling >if J[b) to reajd Russ say , 
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M «, , ." as Someone made/1 g roakfnq Russ say . J M ani (c) to fall to^ 
understand' sliCh expressions as they wheels (Standard English, thelr\car) . 
The use of Imperatives directed to* the reader^such gs. Now dig , may be 

characteristic of black folk tales or BEV, but this- \s unlikely to be 

♦ \ . 

familiar to the white, student. *1 , 

^ Assumptions, The .assumption of re^adabl 1 1 ty'measures^ that It is „ 

J* m \ — — r 

possible to gauge the readability of a text without making reference to 

* r 
the cuRural background and values of the'reader Is dearly violated 

< > - * - - 

by passages of tKls type. Readability measures would not the 

xhi te^tudent's Jack of fami liarity with Black Engllsh,and the con- 

■ ■ ♦ 

**v$ntfons* of. black folk tales. Conversely, of course, a. student who was . 
* to some degree alienated or removed from ynS te middJe-class culture 

* would have agalogous reading <llff 1 cul ties that would not be accounted 

for by read^blllt^formulas/ A major determinant of true readability 

,. ♦ - 

Is the match between the^cultural attitudes, beliefs, and, values of 
the author and 'those of the reader,, " * . 



Job 5t BasalReaders ' ' . 
* ^ ^ * , • 

Basal readers are texts whose main purpose Is'tcfbe- ijsed ,ln teaching^ 

reading,. Most of the basal readers feature ja M contfo'l1erf vocabulary, n , 

• , v ^ - * t * \ 

which permits only a slow, ,pace*d /introduction of >neW words. . They aJso . 
exhibit .simUar controls over' sentence length <anfl syntactic complexity. 
The' standards, for fanguage of basal readers '^Jine Implicit readability 
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formulas, b-ut many oP the series apply explicit formulas as well, In 
el thef^wrl tln£*or selection of passages (ftob Tierney, persona) 
communication)/ * ' * ^ 

* Exanple. The following Js an example from a second-grade basal 

^ reader, (Ginn 720-, Levei'6, Unit 3> see Clymer, Wang, Benedict, 1976) : 

\ t "See the, sights!" called the tall man. 

Every day the tall man came to 5th Street. 

Every day he called,, "See the .sights! See the sights!" : 

P One day Dan was walking ort 5th Street,- 
» * 

Jhe.tall man was there. 

* 

He was calling, "See the. sights! See the sights!" " 
Dan saw,' a big sight-seeing bus stop, on 5th Street. 
There was a sign on the back of the T)us. 
The sign said, "fee the Sights! See the Sights!"' ^ \ 

"What sights? * 

§ m Where does that bus go?" Dan thought. \ 

' * ) * 

, The next day Dan walked up to thfe tajl man. 

"I want to see* the sights," he said. , . * * 

"Mien can I take the bus?" ' ' 

% "You're too Jate today, ,r said the, man. * 

■» "Come back next Saturday. 

\ s Saturday you can see the sights!" (pp. 89-91), * 

* -Problems . This selection lacks coherence, even tboUghJt ts 

' . \ ^ * f , 1 . >" 

supp&jgedly easy to' read according to readability formulas, hi the process 
^ of controjlina vocabulary, sentence length, syni^cjtic complexity j and so 



on, theJ>a*al rfacter authors* h&ve had to ignore other crucial ^characteristics 



* t \ 
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of the text. As a result, a child woo looks for a familiar structure; 

for example, conflict and resolution, will find a story which violates 

that expectation. Laying aside for a moment the question of the 

aesthetic or pleasure value of such a story, one might simply assume 

that the story is harder to read because, of its inadequacies in terms 

f • 

of characteristics such as conflict, suspense, surprise, and- honor. 

Children fed a steady diet basal reader stories 'may, develop a con- 

ception of stories which discourages them , from exploring other' texts 

and which does not match the passages the^encounter* elsewhere . 

Steinberg and Bruce (1980) reported on a study of story dfiaracteri sties 

that is relevant to this problem. They coded stories for rhetorical 

^structure, point of view, conflict, amount of insight into characters 1 

« • 

thoughts and feelings, etc. On^of their findings was a dramatic shift 
from lower-prlmary-tevel basal stbries to upper-primary- level stories 

J fc 

in the amount of inside view (Irrsight into*characters ' thoughts and 

* * • 

feeTTngs). A similar situation obtains with regard to interpersonal 
or internal conflict; the upper primary basals were rated much higher 
on this parameter. Again, Xhere is a maj^r difference between the 
kjnxJs of educational reading materials children receive in lower primary 4 

rades gnd those they receive In # upper prtmary^fades* * Perhaps this 
difference reflects the fact thafe*the, Jower j^vei stories are constructed 

to conform to readability formula constraints, whereas the upper-level 

■ fc 

stories are typically Just ^elected and adapted from trade books* 
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Assumptions* The basal selections show again that higher-level 
factors such as conflict and Inside vitytf cannot be Ignored Jn assessing 

1 * 

readability. It may- well be that chMdren who have not Jearned in 
bas^l texts to understand patternsW conflict, etc., in stories may 
not be adept at dealing with these factors in upper-level texts. 

Assumptions About Readability Measures and Their Use 

The preceding examples have illustrated various, wayi> in whfch' 

readability formulas* gi ve faulty predictions or even lead to the writing 

of passages which are harder to read. In each case, one can point 

to an assumption about the use of the formulas which has beett violated. 

Vfe are led to the conclusion that the fonmJlas are valid only if 

certain\ondi tions hold. Cur list of assumptions has arisen from 

examination of casek where the formulas have fal 15fc, but similar 

lists have been put forth by designers of the formulas themselves. 

For example, explanatory material put out by the publishers of tjie 

\ * 

Raygdr Readability Estimator states quite accurately some of the 

1 imitations, of readability formulas: * ^ 

Jw 

Reader interest level, reader Experience, or any other 

• . 

personal or ethnic variables are not measured by this or 

any i>th6r estimators of readability. Readability estimators * 

« * 
do not measure style or syntax. 



, "Tfiklng materials les$,dl ff Icul t by shortening senteocfcs 
and substituting shorter o.r more cfimmon words for longer 
and more difficult sentences and words may not, In* fact, 
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reduce the difficulty lev^el Indicated when the formula Is 
applied to the new material . The new material may appear 
easier and show a lower gracte level with the estimate, but * 
the coticept level may still be high. Readability estimates 
use variables that predict but .do not necessarily control ' 

* the difficulty of the material . Estimates work best on djs- 

• / ' ' X* 

• cursive or narrative prose. Applying estimates to poetry, 

test items, or other types of npnprose material may produce ^ ^ 

* inaccurate 'results. 

These cautions seem clear enough, and examples such as the ones 

presented in this paper give strong evidence that the cautions should 

be observed. Nevertheless, it appears, that no^ only some, but nearly 

'all, uses of readability formulas violate the basic assumptions on 

their appl ic^bi li ty. The problem is that the assumptions restrict 

readability .formula use to trivial cases of Ijttle* imprfrt for educational 

or social policy. Our exejpples have indicated that readability formulas 

should be used only where the following criteri^are met*: 

L Material may be freely react Material like captioning -for' the 

i * 

deaf, which appears^ the screen and\then disappears after a certain 
amount of time, cannot be freely read.\ TT^e time spent on it is limited 
by external factors, riot the choice of ■ tlje reader. 

£. Text is Honestly written - The formulas ajfume that material 
Is not written to satisfy the readability formulas, but rather" to 
satisfy some other communicative goal. * * * 
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3. Higher-level text structures are irrelevant . The formulas 
assune that organizational material , information about intentions, goals, 
etc., neerf^not be-sp*<^f^cal Jy taken into account. , 

*• Purpose in reading^ irrelevant ^ Skimming, test-taking, 
reading* for pleasure, and so onWe all taken to be> ; equivalent in 
determining the readability of a passage. — s 

5* " Statistical averages are meaningful in individual cases , use 
of the formulas implies that statistical averages regarding both texts 
and readers can provide useful information regarding the appropriateness 
of an individual text for an individual persop. 



6* Readers you are interested In are the same as the readers 



on 



V 

whom the readability formula was validated . Any attempt to expand the 



use of the fo/mula to evaluate materials for readers whose back^ound 
dialect, purpose in reading, etc*, differs from that of the readers used 
in validation is likely to lead to difficulties* * 

Rigorous adherence to these assumptions effectively prevents use 
of readability formulas for TV. captioning, adaptation, selection of texts 
for readers of different cultural backgrounds, designing special texts 

r m 

for cljiidren, sel£«Ion of text passages, choosing trade books, or 

» » 

designing remedial readers, 
i 

Vfe are left with a qqestion; Are there any areas in^which the. . 
assumptions about. the readability fortaulas are satisfied and the formulas 
Improve on intuitive estimates of -the readability of the text? Vfe think 
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not. The real factors that affect readability are things like the back- 

ground knowledge of the reader relative to the knowledge presumed by Y 

the writer f the purpose of the reader relative to the purpose of the 

writer^ and the purpose of the person who is presenting the text to the 

reader, These factors cannot be captured, in a simple formula and 

ignoring them may do more harm than good. . , 

» * 
* Discussion 



Tierney : Publishers of basal* readers are very much involved with 

readability formulas—a frustrating situation for anyone who wants to 

* 

work with a publisher to produce high-quality basaris* The experience 

of Scott Foresman is typical of the dilemma facing publishers of basal 

readers v In the efirly 1970s, Scott Foresman tried out an approach 

called "learner verification of passages, 11 in whioh, as someone suggested 

at this Conference t they actually did go put and try out passages on 
• • " \ 

children. Howeve^, they found this approach to be very expensive, and 

fouid that they did not have the technique necessary to do a reasonable 
* * 

interpretation of this kind of research* At the same time, California 
was developing a policy of checking every single 100-word passage . - 

to make sure it met specified readability requirements. The result 

, f 
was that publishers began to produce basals that wereWery strictly 

constrained— e»q* . a basal at level 2.1 would contain no 100-word 

* m • 

1 » 

passage that scored above level 2*5 on two readability formulas* with 
no consideration of standard error in the application of the formulas* 
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This development wentf Into effect Jn at least four ways; (a) Potential 
. authors received guidelines for avoiding long sentences, sticking to 

/ 

words oo particular word lists, etc. (b) Requests were sent out to 
authors to write within specified constraints, but authors w^h**^ 
ofte'n unable or unwilling to do this. When this happened, the text 
would be adapted, and the author might lose out on royalties if he or 
she wa& unwilling to accept the adaptation, (c) Already published 

/ ■ • - 

< texts were adapted to meet readability requirements, (d) Editors 

sent requests to people who were writing teachers' guides to 

include material that would support the text. This latter action was . 

an attenqpt to minimize possible negative effects of fitting materials 

to formulas, and anyone who wants to criticize the , use of readability 

formulas has to address the fact that the texts do appear In a context. 

including such factors as* teacher interaction and illustrations. 
• * * 

Kantor : Would you say this is true for grades 5, 6, and 7, or only 

. for the lower grades? 

s * 
Tierney : All the grades^ 

i . 
Seidenberg : What is th£ teacher's part in this? What can the 

teacher do with what^Ji superficial ly a poor tcixt? 

Tierney: Typical ly) the teacher will spend soa>e ^time talking about 

a topic before having the students" read a text on it, and perhaps will 

« 

• talk about a few specific purposes for reading the text, 

Rubins ■ 1 think that's a good point, but what does it do to kids 1 
ideas pf what the reading task is about? Reading becomes^ something you 

v 

can do only if you spend half an hour preparing for it. 
* • * 
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Hermon (Linguistics, University of Illinois): Why rely on the individual 
teacher? 

j'jerney : I don't mean to say that I approve of the approach I've just , 

r 

^described; I'm merely telling you the kind of justification that basal 
publishers tend to give. 

Selden : In the publ ishing houses I've dealt with, I've encountered 
less .stringency about readability levels. How strong are the prescriptions 
that go out to story authors? Is there some possibility of honest texts? 

T'ierney: As you move up in grades, freedom greatly increases. At 

rnr ^ ' * ' 

the lower levels, texts are very tightly constrained, and th^ .editors 

probably do most of thd^writing. The real problem with the basals is that 

publishers are responding to popular sentiment rather than to enlightenment; m 

the only time I've ever seen publ i she rs ,pay attention to research is when 

a specific publisher's name* is mentioned in the research. 

Rubin : The amount of enthusiasm an adult, feels for ,a text is readily 

communicated to a child, and it seems likely that a teacher* who has tp wade 

« # • 

i 1 

through three pages of supplementary notes before reading a four-sentence 

text isn't .going to be too excited about the text ,^ancU tf»ls ifill be 

« 

apparent to the 'child. * - * - 

Johnston (.CSft: *Wherf' I # was' teaching , I found some use for readability m 

- • * 

formulas.* namely when a kid would^pome to me and say, ,f l don* thread very 
well, but I'd like a book that talks about this* kind of *h?nq. What 
' can. I. read?" It's very easy to criticize readability formulas, but 
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people who want to get rid of them should propose some other means for 
doing some of the jobs they do, like helping teachers to locate texts 
of appropriate reading levels for children- 

Griffin : I don't know of any research that shows that matching the 

y * t ^ *~ • 
level of a text to 3*<hild is a better way of teaching kids to read 

than not. Carol Chojpsky "V^work, in fact, suggests th?t it is better 

for a child to read a text/that is more syntactically complex than 

his or her own oral jingpage. In general,' It seems 'that hitting above 

_ 

the level of the learner gives the learner a boost up. Maybe if 

yojjMted a good text that was very, very hard, the^tid could learn to 

read just as well as if you l\ad a good text that-tf^s very, very easy — 

I don't know, since the research on this has not been done. 

Brewer (CSR) : That kind of consideration really frwvesJhQ^is* 

though* The^assumption of this conferencfi^ha^TtSf'Been that text 

— * 

features don't matter, but rather that readability formulas ere not a 



good way to measure text features, so the question of what to use 
instead of formulas is a legitimate onef. **" — 

Johnston : In teachers 1 college we were taught how to use Certain 
formulas to. evaluate the appropriateness of materials* 

Zwicky (Columbus, Ohio): Did that work better than reading the 

m f . , 1 

text and deciding for yourself how hard it is? 

1 > 
Johnston L 1 1 ve thought about that, and i t^s true that people's 

intuitions are good at that, but people's intuitions have to be buHt up 



x ■ - ■ 73' . 



.0 



Readability Conference 
72 



9 

ERLC 



You people who have spent a lot of tine studying texts have a gopd tdea 
of the kinds of things?* that have an effect, but a lot of the teachers 
are young teachers who are going out and having ,to deaj^Wfth that kind 
of thing, and need to know what sorts of varlaM^s to look out for, ' 
Maybe you should look to something 1 ij^a readabi I i ty formula as a 
gross measure, and then sensj^tze the teachers tb kinds of things they 
might also look atirr^conj unction with that v , * 

Charjxw: I have a better' sol ution than that. Since readability ^ 
>rmuias don't work very well, you're really better off giving a chtid^^ 
a ^ext you think he might enjoy that may or n*y not be at his level 
and then sifting down with the kid and saying, "All rlghu Did you 
Understand it? Let's ta'lk about It," and in that way having your own 
ability* to judge,.wha£'s a gtod text for a given child. ' * 

Johns ton : That would be 'a good way to do it, but for a normal range 
of well-written fexts at lowei* levels f I don't believe that, a readabil ity 
formula is that bad, 

Zwicky : I <}ueste we're saying that there are not that many well- 
wri tten texts. f ' ~ t 

Johnston: I disagree with that. 

Brewer: One llnfrof attack'against the pragmatic argument here Is 

what Rob Tferney sa t W prtr Cfta't puKTTshe rs out there are takt ng.correlation 
| / 

as causation ahd assuming that the formulas predict comprehensibl 1 i ty , 

so they go out antf destroy the text In order to reach a certain readability 

i * 

level. That's clearly a logical fUw, and a place where we should be 
Involved, ) / s 

;73 .. '. 
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» Selden ; The publi.shirs don't necessarily think that the formulas 

predict the difficulty of the ^adapted text; they thlnk^they have to 

Ttove the numbers In order to sell*. * They'd probabfy prefer to *efl 

texts without using the numbers because it would be cheaper to do. 

Brewer : .But presuma'biy if conferences like this go on and make 

the argument atjainst them, that will eventually filter into the system. 

Tferney : fn some states, like Tex£s, there are state adoption • 

* ' • ** a r 

committees that set up rather arbitrary readability requirements, and 

if textbooks don't meet tjjos^ requirements, they're off the list. 

There are some two million children involved, so it's a. matter of 

a large amount of business,* 

Bruce : Wte trl^d to write a program to apply several different 

'formulas to texts, but we ran into terrible problems tr^Jng to define 

what a sentence is. Different formulas sgem^to have different criteria, 

and there are lots of unclear cases, it sounds like a trivial question, 

but it turns out that a large number of examples are hard to classify, 

■ 

and our progra^rn er f ound that changing the definition of ""sentence" 
changed the readability level oft texts by a grade or more. 

Tlerney : -This sort pf thing allows the publisher to ntenipylate 

* ~ ■ 

the readability of a text by doing ^things 'like changing M tto, . • 
.to "No, . . 
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5. Lowering the Difficulty of Texts Intended for Adults: 

JmpHcat-Jons for 1 ^ la t lahguage" in Legal fcfetifoents 

■ ' * * * 

*- * Veda Char row ' * * 

The study or jury instructions wfnch I diScuss here was not* 

\ * \' 
originally intended asiatest of . readabi 11 ty formulas, but the results 

of the study do suggest that such fornuias are not adequate for 

Measuring the extent to which documents ^such as ^jury instructions are 

understandable. The purposes of*the study were (a*) to* look at the .* 

"legalese" in codified jury instructions and demonstrate that it 

causes the instructions to be poorly understood by the average person; 

^(b) to point out the aspects* of legalese (grammatical ^nstr^cTT^ns 

and discourse structure, not just vocabulary ok legal concepts) that * 

cause the - difficulty; and (c) to demonstrate that if the linguistic 

problems in j-ury instructions were removed, comprehension would in- 

* „ » • • • 

crease dranati cal jy . % • , 

* Jury Instructions 

Jury instructions are standardized rnstructions of law set out in , 

form books, and at the end of a trial / the judge reads the appropriate 

, - # t 

instructions to* the Jury. If the.'Vvrong words" are used, a case nay be 

ap0edl£d; hence, the judge will use exactly the' words^tn tKe book, even 

,if the instruction is "cjCfflcult to understand. 1 ' * ~ ' ' 
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The Docu ment De sign P roject at AIR — - 
**« . . - — » - ■ > « 

The results of the $ttjdy analyzing jury, instructions Indicate that 

grammatical complexity i$ k at the root of difficulties with legalese* 

H«w*ver, many psychologists concerned with language (e.g., Kintsch, 1977; 

Heyer^ J375) have been heading in iy different directiop^ Jbey have 

stopped looking at grammatical complexity, and are instead concentrating 

on such ndt ioos -as* propositi onal density (the number of propositions — 

„ in the form of predications'-- in a sentence). Hov^v6r, the ca?e of jury 

iVt ructions .shews that the study gf grammatical complexity i$ sti II 

very necessary , and that there is a great deal mpre to be learned about 

wliat is a grammatically complex sentence, 

* ~ 

The Document Design Project at American Institute for^esearch , 
(fIR) was set up in -197& under a contract from NIE to find out what makes 

legal and bureaucratic texts difficult to read*, and to help government 

* - 

agencies rewri te ^official documents so tKat they are more readable. 
Hy colleagues at - AIR and 1 haj/e found that the problem of legalese and 
bureaucratese cuts across department Ujies; there is a general problem 

* I 7 * r * 

^.involving grammatical complexity in the texts that are produced By 
.Ubvernment offices. And we have further found that mai?y people who 
can read quite welt cannot read* bureaucratic documents because they 
have trouble/with bureaucratTc language, which may include various 
^orfs of jargon add complex sentence constructions. 

* * * * /► ■ - 

/ ' 
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The Pfelftnlnary Phase of the Study - * * • 

The Jury WiSt ruction" study f which took, place from 1976 through N 1978^ 

a * * * y * . * • • . ' • * . k 

involved looking at jury instructions as discourse, outsTde a trial 
situation. We fir^t chose §2 California civil jury instructions* and 
sent them to trial attorneys, who W£re asked to r^te {hem for. difficulty 
on £ scale of 1 to IK Half of qie attorneys were directed to disregard 
the complexity of <tfte language, and the other half were told to take , 

into account botk language conplexity and legal complexity. It proved \ 

I • ** - • 'V 

to be imp05sJble to separate language from lega^ complex! ty; # tJie results 

.» , # ,.v . • 

were very similar for the two % groupTs, and there was general agreement"* 
among .the lawyers as to what w$s Complex. We then selected fourteen 

instructions tKat would constitutes set -(opening , .middle, .and ^cl 05 in& - 

• : . 1 ' j***-* \ • , • # 1 

instructions), macfe/Sup -a set of ?Tac*s regarding a cgs9 >A and tape t9 1 - 

1 k m % h\ * f~*T "•->'/ s ■ * S^*. ■! * 

recorded the* jury instructions sfl^that tfatK one occurred twice in* 1 



succession. . 1 ■ , . * 

• : • ■ — 

rtie«Task l\ 9 * \ ' K * \ , ' • ^ * ; 

We received permUVlcJn to use« as subjects people yyho % figd been cal led 



for 
orte- to-one 



jury duty, in Prlpte, Georges Qour^ty, Maryland* The subjects, trt 4 

eMntervlew sltu$tnpn, were asked to explain the instructions* 

v » * ^| , 

by repeating them in* their own words. A subject was f (r^t 'grVen a 

picture* of a fictitious accident and a written description qflfthe*- 1 

r "facts* 1 of the accident Cto serve &s'a context for the^ta^Jc) . The subject 

,ther\ heard^ctach jury Instruction t^lce/ aq^Laf ter the ^eoortd playlng^of , 
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aft instruction the subject paraphrased 1 t ofi another tape- All subjects 
were able to perform the task, but the degree of sfcccesS correlated 
highly with £ subjects educational level. m t * 



* 




Scoring the Results 

* ttfstJ&ctions were then broken down into smaller idea units > (clauses, 
phrases) f^r scorjng. .Subjects' paVaph rases y&te transcribed and 

♦compared *o the breakjlowns of thetinstructiojis. "An example of ,an \ ; 

s \ ' ' ' <* ' 

instruction divided into units is as ,fol lows < * * 

9r * • 

- , 1 . U^is my duty .V 

2. to instruct you in the law 

1 3* that applies'* to tt^ts case 

ft.- and you mi^t fbl'-low the law ^ 

5- as' \ state it to you. ) ? : 

6. As jurors f ^ m ,# , b 

7- it is your exclusive "duty. 

8*' to "decide all questions of fact " ) 

r9. submitted to you. ■ *' ' ^ 

i ' 9 " ■ - : 

To determine comprehens ion v of fhese units, we noted whether a subject 

bad paraphrased li't correctly, omltte^ it, or stated it 4^rrectly. 

Interpreting the Jtesults * , # \ * 

^ Vjn^bljjty to- paraphrase of$en i'ndic^ted^a fai lure to understand, 

t * * * - \ - 1 / 

but It could* rfef Vect^otfeer /'actors a$ well, stfch as memcfty overload, 

or the triviality of a glyea idea* We tfed reliable strategies for 

-* ■ n » 

* V ■ " • 

discerning the probable reason for omissions' In the paraphrases. The 
* following were $ome- sppatent sources of difficulty In understanding , the 
Instructions! - , 
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1. Unfamiliar vocabulary, as % in, "A proximate cause^df an injury 
is a cause which, in natural and continuous sequence, produces the 
injury, and without which the jnjury would not have occurred. "J Some 
subjects substituted estimated or approximate for proximate. 
1 „ 2. Unusual placement of phrases,' as in the previous example* Some 
subjects explained the proximate cause a* oner cause jn a sequence, 
apparently misunderstanding because of the unusual placement of the 
V^rase "in natural and continuous sequence." (in reality, the 4 proximate 
'cause starts the sequence.) ~ 

Another example: Given."lf in thfese instructions any rulg, 

< 

direction or, idea is repeated or stated . . . ," people tended to 

y/ I ■ 

paraphrase i-t as "If these instructions are repealed - - - , M probably 

because the phrase in these instructions occurs^ in a position where a 

> 

grammatical subject would have pe en expected- • 

3. AS to construct tons. One instruction used six as to's in 
c los j proximity ; these created vagueness in meaning, and should be % 
reblace^Twith with 'rigard to, about , ^Qpcerning , etc. 

*K .Multiple negatives, as in 'Moment mUrecpTtegtion is not 
unoxwnon. . * .'1 Subjects often got lost amid the negatives and mis- 



interpreted a positive meaning for a negative, amj ylce-versa. ^ 

5. MominaWzatlons, as Jn "after a consideration,^ of the* case with 
• • '< I 1 *' * * 

the other jjurors" .Instead- of "after tionsidering/yo^'ve considered tf<e^ 

with the other Jurors," Subjects' errors Increased arourid such jf 



case 

'4* 
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constructions f as they had trouble discerning who did what. Nomina 11 za- 
tions appear to characterize bureaucratic dQtfjments. as well. 'In fact, 
in some bureaucratic documents there are hardly any verbs at all, 
apart from r$, necessi tates t and fapi ITfates. * , 

<■ i 

6. Strings of attributes, as in, n a witness who has special knowt- 
edge, skill, experience, training, oreducation in a particular science,, 
profession, or occupation. . . . M Subjects remembered only^one 6f two 
i # tems in a string, usually the rgost general ones. ' 

7. Some passives, namely passives irr relative clauses whener WHIZ- 

t m * 

deletion has occurred; e.g., "any Ins inuation 'suggested by a. question 

* _ 

asked a*wf'tness where 'Which is" has been deleted (whence the term 1 

"WHIZ-de^e tion M ) between ins inuation and suggested and between quest ion 

and a&ed ; for this type of structure, 801 of the paraphrases were 

wrong (as compared with an overall average of 50%). 

J - 

8. Discourse problems, such as repeating the same ipst ruction in 

.different words when there was no obvious reason for doing so* e.g.: 

"A plaintiff who is not contributor! ly negligent , and who 

did not assume the risk of harm, and who is _irfjured*as a 

proximate result of some negligent conduct on the part Of a 

defendant, is entitled to recover compensation for such 

jinjury' from that defendant. • \n 

Thus* the plaintiff is entitled to a verdict In this*case if 
you ftnd t in accordance with rny instructions, 1) that defendant 
Was negligent, 2) that such tiegligence was # 'a proximate cause ^of 
injury to the plaiiittff, 3) that the plaint iff was. not negligent 



0 * 
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or f J If negligent, did not contribute as a prbxfmate cause to 

* hi* injury, and U) that the plaintiff did not assuie the risk 

* \ 

of harm," 

Subjects often thought the second paragraph must say something different 

# .from the first; if not, why repeat it? ' 

The Rewritten Instructions 

'4 : : — • - . 

After we had obtained thesj^resufts, we rewrote %he instructions * 
in an attempt to eUminate the difficult constructions and gave them 
, to a second group of '70* people 

.Results . 

Because the general level of education of the second groitp was much 
lower than that of the f i rst. groi*0, each subject received seven original 

# instruction and seven new ones, so that we tfould corrpare performance 

on original and revised instructions within the second group. We found 

\ • • * . 

that paraphrase performance on the rewritten instructions was an 
average of about 50?' higher" than on the original instructions. On % 
four of the instruction* ther£ was no Improvement for the rewritten 

* • V. 

* 1 J 

versions (these had fairly hi<jft comprehension scolres to begin with), on 
a few there was a small^ Improvement r on several there was fairly large 

improvement, and on , others there was 100%. i-mprovemenf . These results * 

I * • ♦ * 

Indicate that. one cannot simply assume that his or her rewrite is , better 

N ' ' * * • * . 

than. f the original; it is n^ces^ary* to go out and tes't it*. 
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Applying the Flesch Formula to the Two Sfcts 

We applied th* Flesch {\S^) readability formula t* the two sets 

of instructions (original and rewritten) and. found that for half the 

.* • 
« ^ instructions, the 'formula predicted- the differences in the right, 

direction, and' -for th'e other half.it did not. For {wo instructions 

that Showed large improvement in comprehension , the readability scores 

<were higher, and for one .there v/as "no^rfif ference. For .the four 

instructions on which there was no difference in performance between 

the original and rewritten versions , tfie formula showed better 

, '.readability. Hence, the readability formtfl a was»i steading,- as it 

>- 

made the wrong prediction in half, the cases. 



tonclusign 

/ 



* Cphtrary* to what is usually'done in rewriting, some of the Ranges 
\ \ 
we made in the instructions invojyed increasing* paragraph and sentence 

: i 

, length for such purposes as adding more context* Subordination patterns 

were also changed, (e.g., eliminating left branching and center embedding}. 

Thus\ the rewritten instructions may have Contained many long sentences 

'{which a 'formula woul^te less ."readable") , byt ttie sentence structure 

was siirpler and clearer than the long sentences^ the original 

* ' * / / * • 

instructions. (The Document Design Project, 1979, has incorporated 

* • *■ * 

several of our findings into, its -17 guidelines, for langu>ge slmplifica- 
* ' tion). 5 * 



♦ • : ' . . • 83 



V 



-v v Readability Conference 
82 



Di scussion 



v 



Seidenberg : Perhaps for lawyers legal ese is perfectly clear and 
comprehensible, so maybe^ it serves a g^^ne function. 

Charrow : While It would be nice if this were true, apparently it 
isn't/ David MeMinkoff, a law prfcfessor at UCLA, claimed in a recent 
paper (Note 5} that lawyers do not understand legalise very well either. 
As an example of the difficulties lawyers run into, the word sh'ali *ls 
generally favored in ie§al writing instead of must , leading to ambiguities 
Between tfie future f tense and the "not ion of obTigatTon or necessityT~~ 
'Lecjalese is perpetrated because of reusing parts of oid contracts , . . 
* Selden : *The legal writing courses that law students take . « / 

BreV/er : And the desire to maintain a specialized language that, non- 
lawyers will have difficulty understanding, 

Kan tor : It would seem ta be desirable to have positive guidelines 
rather than simply telling writers what they should not do^ since 
telling them what rfct tp do doesn't necessarily help them do something 
else instead! • } * 

Charrow: Many of khe Document resign guidelines are formuj^ted 
posi tively—e.g/, "Use the active voice" (this particular one, is 

motivated by the need not to leave agents unspecified, rather than by 

* ■ 
grammatical complexity)* * 

• Selden : The Army has found that guidelines per se are not too 

■ * ■ r 

useful, since writers cannot keep them all in mind while they are writing; 

It is better to give examples of good writing, along with explanations 

of particular points, v • * y 
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) Charrow f The Document Design Project hej^s teach good writing by 
offerfnjj courses for bureaucrats- i ■ 

Some Possible HoPalsJbf This Research - \ 
: > " 

Davison : Jf the researchers had simply used readability formulas, 
^^^^^^^^^^ p ' 

illegitimately translated into guidelines, they would not have been able 

'to achieve the remits they did on the jury instruction experiment. 
* I . 

v Successful rewriting is very difficult and expensive and requires people 

who know what _t hey are doing--/K»t the mere application of a formula. 

Anderson: If the/ researchers had used only one readability formula 

■ V 

to decide whether their rewrites were effective or not, v theyVou^d have 
been misled in this case. ^ 
* * Chanw: When dealtpg with legal language in particular, readability 
" formulas cannot be used because (a) it often is impossible to get away 
from the type of (legal), vocabulary needed, and (b) long sentences may 
be necessary, like many of those in the rewrites. If a more adequate *• 
formula were devised", probably nobody but a linguist would-be able to ■ , 



use i t. 



6, Rl&dability Formulas and the Adaptation of Texts * . ^ 
Alice Daviso^ and Robert N. Kan tor - 
Readability formulas were never intended by thei/ originators to 

t 

serve as directions for writing a text to a part|cular grade level or for 

* ■ . \ « 

simplifying a text so that its level of reading difficulty is. lowered. 
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Writers 'oft readabi I i ty formulas usually aJd a discla^er to this effect 

ft „ \ 

(e.g.j Klare, 197^-75); and properly so. A distinction should be made 
between measuring factors correlated in 60me 'way with reading difficulty, 
and actually defining th$ factors which cause a text to be difficult 4 
to read. ' 

But readability formulas involve only a few such factors correlated 
with difficulty. On^e writers know how readability formulas work, it is 
hard to forget that they involve sentence length and unfamiliarity or . 

length o£ words-r«both 'pen rosivo fe atures of texts. And even if writers 

k f * 

are not told to use short sentences .and simple vocabulary, they might 
Independently conclude that doing so would bring the text to the levef 
of reading difficulty which is desired or necessary for -the particular 
purpose for which the te^t is to be used.. 



Adaptations And Sentence Length • 4 

In this section we want t6 present some particular cases where 
evidence internal to the texts in questions leads us to believe that 
readability formulas have had some influence or> the writing of materials* 

. ,sj V ' ■ 

intended for reading practice. The examples are excerpted from a longer 
study (Davison,, Kantor, Haqpah, Hermon, Lutz, £ Salizillo, 1 980) whe^e 
we systematically compared the original and the adapted versions of 

W i 4 • 

■ . * 

four texts in, Parker ( 1963X- There we argue that many of«the most 

j, m 4 * * • •§ • 

- - * 4 ->^ 

f * • » • *> 

appropriate and successful changes Could not have tieen /nade if the , 

' • , !'«*'. 
adaptors were -using readability fofnxilas as their sole guide \n "Kfewrl ting. 
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Vfe^also argued, as we do here, that therewere ether changes which were 
not particularly successful per se; or which rn fact made the text 
harder to understand. These changes did not have., any obvious motiva- 
tions, but stnee they involved Shortening sentences and changing 

' ' ' *' V 

partX^ylar wprds, it is probable th^t they wereHnade with'readabj 1 i ty 



formulas in mind. This feeling is supported by a 'comparison of the 

• ' J 

reading levels of the adapted texts with the original versions, as 
asu/ed by a number of the standard formulas .(see Table 1). Put 



ir^a 



Insert Table 1 'labout here. 



v — v-* 



crudely, it looks as though certain changes vfere made to those elements 

4 

of the text [which are measurfed 1>y readability formulas. 

So if the reading ley^l of the texts had. to be a certain figure, 

the adaptor could count on this figure being reached by making sure 
< * • 

that there were few vei;y long sentences, that mo$t sentences di*not 

Exceed a given length, and that' the vocabulary was.neKher unfamiliar 
* • « 

nor complex ih length.* 

• M 

, Adaptations and Sentence Restructuring . > / * 

m m 

An overall sentenee-by-sentence comparison of the Original and 
%v " * * 

, adapted forms of- the four texts stuped did in fact show that there 

* — I ■ . S / 

were decreases^! n the average length *of •sentences as measured In w6rds ■ 

> 

(rather than sy I lables*-see Table 2). In the adaptation of the Jonger 
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texts, a lot of. material was simply deleted so that the absolute nun^ber 
of words (and "sentences) was dec/eased (see Table 3)* Sentence length 



Insert Tables 2 and 3 about here. # . 

- — ^ ! 

was decreased either byjoutright deletion of mate ria 1 Xvhi di we -assume, the 

.adaptor felt could be left out without distorting ther\e/t, or by breaking 

, up longer sentences into their constituent clauses whic& tbfen. become 

independent sentences. Table A shews that the number, pf long sentences 

* 

with three or 'more clauses is substantially reduced in. two of the texts 

studied (two whose original forms were of manageable length for counting 

fr- 
et auses by hand). The number of sentences with one of two clauses was 

somewhat increased* Tables ! and 2 show that thfe number'of words in the 

-adapted texts were distributed antonc^ proportionately more sentences than 

v 

in the original, so that the sentences were simply made shorter. 



Insert Table 4 about here. 
— -t" 



One effect of shortening sentences is to introduce uncertainty Into 



the text. Splitting S complex Sentence Into Hi component parts often 



rfeq^'es "that the adaptor delet^connecti ve" worfds^like subordinate 
conjunctions or felse remove grammatical markers lite the infinitive 
particle to, which are not found in independent sentences, but con*' 
jurtctiohs. and other clause connectives also have meanirtg, and deleting 

w * ■ 

them robs the resultant sequence of sentences of some of the author'^ 
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original fotent, particularly Information about the logical relations^ 

between the sentences* 

• • - - ' * > < . ... 

Some examples are given below. * Jn (I), ^"original sentence has t 

dn infinitive pappose clause, which Is split off from the maio clause. 

o 

In the adaptation, * - * 

4 1 ) ( Disaster lf> Dayton ) 

Original: "I'm going down to the contract," said Jack", 
; "to'see-that everything is all right." 

Adapted: "4 f m going down to the building project," said 

. Jack- "I h^ve to see (f everything is all right." 

The altered version is a statement of obi iget ion* not of purpose, which 

fould not be expressed as a main clause without redundancy (e.g./, 

I'm goinfr^o see . - -)- The expression of obligation might allow the 

reader to infer what .Jack's purpose- was in going, to the building 

* $ ' •» 

project, but the change represents a distortion 'of the text in that 
* >. . * 9 \ 

the semantic/pragmatic notion of obligation Is not described inrthe 

original and F s not just i fied in the ^adap.teUion . ^ 

' ' * \ * * * 

In example (2), the reader must also Infer the correct relation 

between ' two clauses i^n the adapted version; ** * c • 

* (2) ^ California's Ciants) " • \J * -< \ 

OrigWal; If given a chance before another fife comes, the tree 
will heal fts own wounds Jjy growing ^ew bark over the 
burned part . • m 

Adapted: If cjlven a chance before another fire comes, the^tcee 
* wU^TlfiSaJ it* ownj^ounds. It will grow new bark^pve^ 

the burned p^rc- * ' 
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The,, reader couliMnfer that growing new bark is the means by whfc^ trees 
hearl^their own wounds if It, is not explicitly ^stated in the adapted 
verjsibn. Y^t an inexperienced reader, or one who does not 'know very 
much about trees, might* make an incorrect jguess and see healing wounds 
and growing new bartj as separate processes, simply ordered in time. 

These are quite clear cases of loss of relational information which 
is explicit in the original and only partially recoverable from the 
adapted version* Some deletions of conjunctions s % uch as and may 
simply make less l clear -which, sentences have related topics. But 
Qther deletions, while not removing much explicit relational informa- 
tion, may take away Hints as -to what causes what- \ 

For example, in splitting up , the long sentence in the original of 
(3) into three ^separate sentences, the adapter has left fewer clues 
for relatir/g the clauses. The connection of the first two would convey 
tQ the adult reader that the narrator wak trying to keep from being 
"afraid through J:Tft tttgb^^&i^ th&. dangers wer6 not vi si ble t and nothing 
much could be done about. thep until the, dawrT^tnr^Hmit-^_the 
narrator's conscious effort. The adapted version simply describes a 

sequence, where the codling of the dawn and looking out are related only 

% ■ * J . . 

in time to keeping up courage, * 

. (3) y ( Disaster tr ^yy} . 

Original: I had kept )ny nerve pretty We 1 1 til dawn, Just as 
„\ , the faint 1 ight «was fcomlng, when we looked out and 

. *saw the water whirling by, ^gainst the bay window* 




A 
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Adapted: But we all kept oUr co^rpge up. As the .faint Ug^t ^ . ■ 
- 1 of 'dawn V/as coming, we, looked <5ut. The^witer was^' 

- * whiWihg by.; ^ * ^ 

Splitting. up the third dnd^Fpurtfi clauses Into two separate sentences 

; (which Ts the resulUof deleting ^and saw) - fliakes" the connection between 

the ideas or propositions ijife cable but not explicit.; Clearly, the 

emotioQal o^rtoncr'of the 'Vvhole,^ as expressed by*" the particular clausal 

• ' * k - * - 

combination irt 3 (original) has been obs^Jrerf^-" *^ * , 

■* * - 

• Deletions of Point of View and Modality" " " c " • * - 

The particular loss o£ Information 1n-this case is' also found in 

many other examples where verbs of perception, Speaking, or mental process 

\ ' * * ' ' * . • * " ' ' <; ' ' * ' 

°are deleted. In the adapted version abov.e., the statement The water was 

whirling by could be correctly attributed tb.fthe narrator, s(nc$ there 

is no distinction made between the thoughts. and perceptions of the narrator 

and fhose of the characters in the event . **But in the more "descriptive 

texts, there Is often a great difference between thfej, ideas -of the writer. 

: ■ ' * r * * ■ - 

and those of other peopJe mentioned.' Learning to. distinguish between 

* " * V/* * * 

assertions of the writer and opinions attributed' to others, npt necessarily 

- . 'I \ -V . ' 

^KaTed^by t^he writer, is a ntojpr Jas^ for an inexperienced reader "and 

I • * . " ^ . 

should, wfe believe, be a majot Irtstrjuct ional^goal / However, the adaptations* 

> we Studied *cons?.stentjy seemed tt> remove 4 material flrom sentences which 

t gave information about the source of a statement or which made a statements 

>' - | t , : "/^* * < "/ 

seem as thpugh it was not asserted cateWrically by* the au^bor'.^ 

* *M f ^ •» 
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.Information about the source of an idea 1s deleted when the 
proposition in question is nat disputed by the author, as example (k) 
illustrates. " 

» ft* 

< (k) '( California^ Giants) *■ ' ; 

/' 

Original; A railroad freight agent has figured that it -would 
■ « 

require at least kO modern flat cars to haul aWay just 
* • 

tKe trunk alone. « 

* ■ • • 

Adapted:* And at least -forty freight cars would be needed to haul 

away just. its trunk. . 

• ♦ « ." . 

The assertion would be more convincing it seems to Us, if its source 

is described as a person who woultf know (perhaps, better than the author) 
exaqtly what an enbrmous task such as this would require. Similarly, 
the example in (k) illustrates the deletion of a description of c% source 
(5) (Milk) ' ' ^ ■ ' '* ' 

, 4 

Origina): Romans were said by Pliny to* rub bre^<f soaked in asses 1 

r milk on their faces to make them fairer and preVent th§ 



I 



, " growth of beards. 

Adapted: The Romany rubbed bread soaked p Jn assek 1 milk on their 
•* * * * 

^ faces. .They thought that thi s'tould make their skin 

paler.. They qlso thought it would keep their beards 

, from growing: # K * 

Thq assertion about the Romans ought to gain credibility by toeing , 

attributed to^an eyewi tness , but this information is ^Jeleted, perhaps 

because ttfe^riter would have to describe who Pliny was. (A paraph>ase 

might have been used, such as M a writer who lived in the Roman period 

. • ■■)• In the adaptatipns the writer adds They thought - % . ,*to a 
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proposition which is supposed 1 to _be ,.uatXue t _though-4a ordinary ^dtrlt 
writing, the attribution of a proposition to a source does not indicate 
that the author disagrees. Instead, the reader must learn to judge in 

a particular context whether the author agrees with cited soruces, 0 * 

J r, 
disagrees, or remains neutral. ' * % 

. The deletion of the source of a proposition has several adverse 

effects, in our opinion. Most important, it obscures the difference 

between fact and opinion; statements are made boldly, as though they 

# are simply facts emanating from an infallible and author! tati ve^sourc^, 

the author. Secondly, readers ar€f given no practice in distinguishing 

* shades- of Qpirtion, deciding who thinks what, and in evaluating the 



probable 'degree of truth*which a statement may have., This same effect 

V ' 
is found in tfje "deletion of modal information, as in (67: 

' (6> (MiMO" 

f , * . 

Orignjal;- Nero's wife, Poppaea, took a daily bath in Jt (*m x ilk) 
and supposedly had 500 beasts on tap for the purple. 

t Adapted: The wife of one.emperor of Rome took a milk bath every 
, * day . - She kept five hundred, animal s to make sure 

— of haying enough milk each day^ 

The' adapted version presents ^he proposition as a known fact, while. the 

original presents it as probable or conjectural, given that it is a a 

fact about a remote period known through possibly") unreliable sources. 

A third and perhaps, less pernicious effect of deletions of source 

information is simply to deprive the texts of some color or Immediacy." 
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(7)- ( Disaster in Dayton) r 
Original: J gotjced the water kept rlsjng. * 
* Adapted: the water kept rising. 

Since the narrator and the protagonist are the -same, ther^is na chance 
f.dr the reader to misunderstand the soufce^ yet still the adapted ' 
version conveys less vividly the relation between „ the events and the 
narrator; in (7), the hare statement that the waters were rising would* - 
make the reader less aware than would the origi/ial version that the event 
was likely to*make the narrator Still more apprehensive of danger. 

-The net effect of all these deletions is to make the sentences in 
the adapted ^version of the text "much shorter tHarivthe corresponding 

9 ^ 

sentences of, -the original. The primary .motivation to .delete this 

& ' " ■ ' - • 

material seemi to* be to shorten the text, and since the deletions do 

nonradical ly distort the meaning of the text, thg losses of informa- - 

^ion are considered tolerabfe by the adaptors. But, as we "have 

demonstrated tn the preceding examples, the loss of information in 

sentence connections, may make some of the meaning of the tdxt harder 

t# recover phan in the original (examples K3). Deletion bf information 

k fe 

about »the source or probable truth vajue of a proposition, as in \k) ~ 
(7) i 9* vcs*th"e*inis leading icnpression that all assertions are eqtially 
solid facts and takes away the opportunities for" the reader to learn 
to make judgments about the reliability of an asSertioft. 
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lexical Changes , 

* * 
The changes made in the vocabulary^ of aJ text may also lead to loss 

\qf infonnatio^TTvajnor^ familiar or shortei\word is not available as 



a ctose counterpart of the unfamiliar/ 
^ surplus is deleted* In addition to ij 
W (Milk), 



Jn the* fol lowing example, 
[ion about quant i ty » 250 gal Ions 



/ 



•Original: In Toronto* a suburban ice-skating rink was flooded 



with 250 surplus gallons of it ^ skim nrfilk)*. 
Adapted: An ice skating rink was flooded with it. 
The deletion of surplus takes away information from which the reader 
.might guess why mi Ik was being used in this apparently wasteful Manner. 

A paraphrase to that effect might have been more appropriate: « . ; .was 

( ' * H * - 

flooded with milk which was nfrt needed ; mhgftt have spoiled , etc* .put 

such a paraphrase would have lengthened the sentence ^and so might have 

been undesirable if the .writer was alsdbnder thte injunction (tacijt or 

otherwise) to reduce* sentence length* 

Very often the^gbstTtution of shorter or more familiar words rpust 

^ tead to lengthening of sentences. An extreme case is MJustrated in (9) 

"(9) • (Mi 

Hippocrates recommended mi Ik as ,a curat iv^ beverage. 
7 words) ' 4 

ne. of the most famous Greek doctors told* his *patients 
to drink milk to cure i a llnes$, 06 words) 
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The adapted version is lengthened by nine words to ,explairj^who Hippocrates 

was (saving one word by not mentioning him by. name). Curat iVe , which is 

the least familiar word in the sentence (according to Dale S O'Rourke, 

' \ I 

^1976) is paraphrased in three words', to cureTllness , whi\e recommend 

and beverage^ neither of which is very unfamiliar, are combined as told 

. his patients to drink . The adapted version does convey the inf<(rm^tion 

of the original, but in simpler terms and at the cost of lengthening 

the sentence to more than double its original length/ 

/ ^ Many words are given simpler substitutes even though the original 

* - 
words are Ukejy to be familiar to readers of the intended grade letfel 

though perhaps not guaranteed tob? familiar to readers reading below 

« 

grade level. Since the texts we analyzed were intended for students in 

the seventh through tenth grades who read at the fifth to sixth grade 

level, it is understandable that all less farg 1 liar words might be given 

substitutes, paraphrased, or deleted in order to remove any possibility 

of baffling the reader, even if the text itself did not benefit by the 

f> change, - . 

It is'alsp interesting to fjote that the Adaptors in some* instances 

f - f 
went farther than considerations of readability formulas would demand. 



They carefully changed all obsolete, idiomatic, or colloquial (slang) 

I ' " / > 

expressions which might not be in tonroori use, substituting, 

for examp-le, the term bul lding project for contract in (1) and courage 



for nerve In (3). *They also deleted unfaml 1 iar names, like PI iny Id«(5), 
or gave an explanation of who the person was, as for Hippocrates (9) and* 



wk • •• ■ . • . • ioi 
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Poppaea In the context Tfor (6); she Is the wife of a ftoman emperor . Such 
changes shew a sensitivity to the difficulties of the text, which do not 
follow from readability formulas, s I n<;fi«. proper Viames. are not supposed to 



enter into the calculation of vocabulary complexity In many readability 
* * ■ 

formulas* - • , 



Consequences of Formulas as Writing Guides 

Using readability formulas as implicit guides to writing or rewriting 

thus poses the dilemma that tn ordqr to simplify vocabulary, the writer 

'is often forced to lengthen sentences; but to decrease difficulty of 

\ 

sentences, the writer must also shorten them. One way to deal with this 
problem is to delete information^ but some of this information may be 
useful or necessary for interpreting the relations between- sentences , and 
without it, th^r sense of the text is not preserved, ^ven if some connec- 
tive information is left in, the separation into independent sentences 

treates the effect of a sequence of unrelated assertions. This effect 
' * ■ ' ' > * 

may be Seen in the comparison of a sequence of two 'sentences from the first 
edition of a* text <Bendick £ Gallant, 1980) with the same passage from 
the sedond, edl tion, where a subordinate clause has been made Into an 
independent sentence:*, ' M 0 • 

1st editiop. You probably saw Illy pads, grass, reeds, and water 

1 wfeeds growing In .Shallow water near the shore * * And A 

" * \ — ~ — • 

_J , ^ * maybe A there Vene Water striders gliding" over the 

surface^of the lake, and small fishes darting among 

the shadows, of the lily pads. 



4 



* 1 
2nd edition. ' You probably saw lily pads, <jrass, reeds .and water, 

weeds. These plants grow in shallow wateV near the 
- shore- There may have been water striders gliding 
over the surface of* the /lake. _ 

J 

' ■ (Bendick S Gallant, 1980) 

■ f 

■ 

The subordinate modifier growing . ■ ■ has be'en made a separate s'entence 
"connected to the preceding sentence by the anaphoric expression these 
plartts. But as a separate sentence, it appears to be an ^assertion on 

— r~ 

an ecjual level in ^i-scourse with the preceding, rather than just inter- 
esting background information. The next sentence about water striders 
irefore'appears jtota! iy irrelevant and inconsequential, because its 

^ """~""\ 1 

. connect i on \with the description of the lake i-s lfcss cteai-than in the 

• • * ! " ■ - * * 

original. V 1 

Changes such as the ones illustrated in the above examples generally 
seem to be dictated by the implicit injunctions, following- from reada- 
bilify formulas rather than by genuine considerations of difficulties 

"^wKbJn a particular text^ We have noted that unmotivated changes lead 
« ■ • ■ 

to Joss of information or" create poss.ibi 1 Hies "for* misunderstanding 
the original content of XH^ tex.t. -It would seem more productive,, as 
we have noted In Davison et al. (^80) , ito- alter the text where its 
actual properties make it necessary. Fpr example, .the relationship v 
between the negative and the rest of the sentence in. (il) is fairly 
difficult to see, because the negative does not. just simply negate the 



verb. ) 

* * - 
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V 

; 

V 

(11) (California's Giants) 
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Original: When a Big Tree, f a 1 T s > its heedle-like leaves do not 
wi ther for years . 

Adapted-; — -Wben-a^W^-tHPee-f al Is , it takes-y^ars for its needl£llk&_ 

• k 

leaves to wither. 

t 

Tftat.is, the leaves (to wi ther but not for years. 

No readability formula would define this combination as difficult 

" , . • ( 

to comprehend. Ox? 1 y a writer us tag knowledge of language would have 

defined it as misleading and made the appropriate change. 

Reasoned Adaptation 

01 ear ly^ an adaptation of a text which js done for tha^urpose of 

making it eas/clr to read will involve simpl if i<^tion and substitution, 

and probably also some reorganization. Tf this work is done with . 

• ■ ■ 

seme sensitivity to .proper t ies of texts and to the expressive 

/ n * * 

characteristics of words and sentences, and if the intended audience, 

actually does understand it, It would seem that readability formulas 

ar*e basically unnecessary. When readability formulas do have influence 

over writers. We find such influence tends to undo some of the work of 

adaptors, as intelligent writers by causing fchem__t^ )ess fc 

(Connected and i t smear^^ seems to us to bp a waste 

TrTteT^efforts to ask them to simplify a text l?o t th according to 

their own knowledge of language and according to the contradictory 

injunctions of readabi 1 i ty, formulas . v * 

\ 
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, 7. * Comprehension of Captioned Television 

* i ' 6 ' * 

a Mark j&eidertberg * * 

Virtually all of the 13-4 million hearing-impaired Americans (Schein 

■ * • * - 

&'Defk, 1974} suffer some degFfe of informational deprivation. For a 

« * ■ , 

small percentage, thei v r handicap merely reduces -the flexibility and 

value of , their TV viewing .rather than excluding them totally from the 

medium, for most, the ^^nsequencef^e more severe. Little TV 

programming other than sports can be understood more than superficially 

* B * 

without the auditory component. Lipreadi'pg provides little additional 

« 

„ • * 

information because of the small picture size, poor image definition, 

• * * 
and unfavorable camera angles- The cultural and informational 

* ■ " r 

isolation that results «js formidable. " • 

Comprehension of televised information can be Anckeased by the^ 

y * ■ 

addition of either captions or s imuItaneoustjcaos+atTon of the audio 

' • — ^ x « ' 

into sign langujagt^ the f^frored method both because many 

JJjearTTfij-Tmpa i red individuals do not know sign language and because, 
among those who do, captioning js greatly preferred (Norwood, J97&). 

The recent development of "closed" captioning via Line 21 of the 
broadcast signal and ^ts implementation by thi%& networks (Browa, 1980) 
ensures that the, amount of captioned programming will increase greatly. 
In addition,, full-page text transmission such as the Teletext system 
will sooh -become avc^lable. Thus, an array of Innovative electroni c 



text^'servtces is likely to be available-soon 



/ 



i. 

\ • 



. ion 



It thus becomes important to understand the cognitive and Mtyjui 
9 \ 
r factors involved In caption comprehension. This paper reports the, 

results of a £t\idy of current captioning praptJces. Transcripts of 

programs captioned by WGBH-TV (public televislort^in Boston) and video- 

\ . . } 

' tapes of captioned programs were analyzed along several dimensions. 

-v 

Our preliminary findings suggest that captioning for adult viewers is 

currently done primarily on an intuitive basis* Little is known about 

t 

the properties of captioned texts that contribute to their coirpre- 

hensTbility or about* the consequences oJF curren£--f>racTTw In the 

simplification process^j^ciies-to ^fextst rue ture, connectives, and 

N ti£0$4~tT0naT material are frequently deleted*- This may resiri 

te*t that is syntactically simple but difficult to comprehend because 

the reader must generate deleted information and integrate different 

portions pf text ih-the absence of explicit cues to its^ structure. We 

jiave isolated five groups of factors that appear to'heavily influence 

caption comprehension: 

]/ Display Cc^venttoris Including the division Jif captions into 
lines and successive displays; ■ > * 

2. Cues to Sentence Structure such as relative pronouns and^ 
m K complementizers > which may be retained or deleted;, 

3. Global Structure Cues such as* introductory sentences > _ 
transitional words or phrases, and summary ^material ; 

„ k. Lexical Choice including t£e retention, ^eletlon,or replace- 
ment of non literal expressions such as metaphors and idtoms; 



-1 — 
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l t ' . . 

5. Genres of program content; for Instance, short and long news 
pieces, "teaser 11 . stories, feafiires^, drama, " „ 



These should be -the focus of futufe experimental 





irevtops Research 

> 

(fsting research indicates that the televisipn viewing ha&its of 

Tearing and hearing-impaired persons are comparable. For example, an 

evaluation of the Captioned ABC News performed by th^Oeafness Research 

and Training Center at New York .University fourxkthat among more than 

X 

1,100 hearing-impai red respondents, viewing averaged about 3 hours 30 

minutes per day. This compares to 3 hours kk miputes among men in the 

genera) population and 4 hours 51 minutes among women. Sencfelbaugh 

(1978) -found that hearing teenagers watch about 20 hours of television a 

■ « % 

week, while he^ring-impai red adolescents watch about 30 Hours and deaf 
adolescents about 36 hours. 

« * 
Little research has been conducted on captioning; most studies have 

- ■ 

focused on thei r general effectiveness. Dayila (1972), Fischer (J£7'),'- 

Gates (1970), Nix (1971 and Norwood (1976) showed that televised 

i _ _ * - - - — 

transmission of Information to he ar Ing-impbi red subjects is improved / 
by the addition of captions. Propp (f972) £ound that captions were , 

the most effective of four presentation methods (including simultaneous 

< 

sign language JnteVpretat Jon) • A formal evaluation of captioned- 

4 

"Zoom" (Wfnslow, 1977) found that children viewing captioned -programs 

•* ♦ - • 

had consistently higher attention level* than children viewing uncaptionpji 

■ • • • . *' . v '' 

programs, ' •■ 

* r ' / 
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, There has been sonfe research on display factors. O'Bryan (Note/6) 
, found that a regularly paced flow of medium- length two- line, captions 
seeded most efficient and least fatiguing for deaf readers. Consistent 



cap U on placement was also HnpoTtSnT* and hr^ras f ound^ that captkSns 

. * ' • ' ' ' # \ . 

should be shortened over active vi^eo scenes and never cover the 

speaker's mp>uth. O'Bryan (]975) found that on color broadcasts yellow 

* a * 

captions were not 'as effective as white Captions. 

In terms <^f linguistic structure _and content, the research is % 
sparse. An evaluation of captioned television for deaf adults (Bickman, 
Roth. Szoc, Normoyle, Shutterjy, & Wallace, 1979) showed inconclusive 

* " f *v ■ • 

•results regarding the effectiveness of edited versus, verbatim 
captioned scripts. The study suggested that 'hore research Is, necessary, 
to examine and f uf ly delineate the important linguistic link between * 

the captions and information transmission." 

* v 

In sum, previous^research has shown (a) that hearing-impaired persons 

watch television, and. (b) that captions are beneficial., * However, almost' 

; *> * 

no research exi sts concerning caption content. 

• • ** 

. - - 

Issues 4 °. * < 



_ * Thejoal of the present research Is to identify variables that ^ 

affect caption cyprehenslon in orjler to develop guidelines fpf ctfpttoo_ 

wrtjtfng. Caption* comprehension is a language experience that^tfraws 

upon skills used In listening and reading, yet it differs considerably* v 

V * < ' 

from.these tasks.' It is similar, ttf listening in that the rate of 
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* \ * . * - * 

" transmission is controlled by the sender. <£t caption is ajso fast* 

fading (Hocketf:, 1963). * The actt/al signal is preheat for only a few 
- seconds, and a £iv,en caption ,must be' linked to previous information 

r *?, * - * v . - - \ .< , V- ; ♦ 

that Is nop pnySJcaJly available y - Captidk co'rrprehension Is sidiW 
f to readjng, of course, In that the channel bf*"ptfrrrnunfcatioi> is, visual, 



and the medhjm is wp£ tjbe^lkflguage. It Is a form of reading^ hbwe'ver, " 
.br which.* lookback? are impossible. In addition, the cations -compete 

with tile accompanying video for, attention. f ' ' 

p-^ . " ' • * * 

* / * • i 

The main goal in captioning is to present captions that are easily 
- understood yet accurately preserve as ntuch of^ the jnformation in the 
audio ( as*, possiiyfe. Under 1 .current procedures, Captions are simpjifijed 
with respect to the audi^for two primary reasons; j(a) Whereas audio 

information may be' pro^ssed^lj^^Itancously ,with the video (because 

\ * ' " ' I 

these sbirrces are-ih different channels), captions share the limited- 

\ . % 

capacity visual channel with other infatuation; and (b) the target 

audience, ts assumed, to- have low-level reading skil.ls (Conrad, 1979; 

Quigley £ King, 1975). Captions -are syntactically and Te*fcally sinpler 

i" * *. • • * 

, than the. original aJdfd; fdifcmatic expressidhs -and metaphors are replaced 

. / ■ * % * + 

pith concrete language,. In effect (although not by design), Qaptloners 

are following strategies. Vhich increase the M rea<labi 1 i ty M of a>captioned 

te'xt as measured by standard readabj 1 lity formulas /e.g/ ,* Bo/muth , 1966; 
^ - - . * * 

Dale & Chall, 19*»8; Ftesch, 19*»8)- In addition, portion? of the audio 

\ - 

are not captioned at al). # * * ' 



* 
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Jhe effects of these simpl i f I cat Ions on comprehension J retention, 
interest, and enjoyment are largely unknown. % Preliminary analyses 

of several captioned programs Indicate that, many current procedures 

\ , • t *'* 

may have negative effects in these areas. Cons-idef t tb$ following u 

example, in which the audio portion of a news broadcast is compared 

to the captions that actual ly appeared on the screen. 

" * I V " 

Original --(fteporter) : The energy proposal almost went * 

down the draip today here in- the House*of Representatives. 

THe jb^'ttle was, over lumping* the naturaf^gas bill with less 

controversial energy bills. Some senators wanted to splltr 

off naturaf-gas, hoping then to kill it. * 

(Senator) : "J could gjb on bM day reciting the prestigious 
rrational organizations that aVe in all-out opposition to this 
legislation* And yet, we're being toy as a proud legislative' 
1 . body that we should swallow this whole indigestible mass with * * 
one single up -b r-dowii %g t e J 

, s CaRti'oned versjpl^ H Miborter) : The energy program almctet 

ended today irr the Hobse. bf. Representatives. Tke argument Yras 
§ • * ■ t ♦ ^ 

^ about linking £he natural-gas' bi 11 to less controversial energy * 

bills. > * . ' 4 . 

^Senator) : "I could continue listing the honored national 

Organizations that completely oppose this legislation* But we 
- 

are being told that we should accept this whole legislation by 

* 7 
' voting on it once* 11 ' 

m% The captioned text differs from the original In several ways* Wt 
has been shortened b^about a ^hi^^t^rough deletion; Individual sent- 
ences are shorter and syntactical ly less complex* Simple lexical items 
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\ ' 

have been substi.tuted 4 for more difficult one^ {e.g., '"honored" instead 
of 'Vrestigious' 1 ).* Tdiomatic expressions have been 'replaced by non- 
idiomatic ones {e.g., "end^d" replaced the idicro '*down the . drain'*) ■ 

These alterations, which are typical, affect both the ease with 

- — % " 

which individual sentences can be decoded and the coherence of the 

story. Some alterations change the meaning of the text; this is 

especially true. In cases of ic^om replacement. For example, saying 

that M the energy program almost went* down the drain" does not toean 

''the energy program almost ended". rl The latter implies that an existing 

energy program was aboufc* to end, whrth"is untrue; the former correctly , 
• * ■ 

suggests' that the bill wtiich would have t created the energy program 
was almost defeated. Similarly', idiomatic phrases in the quotation 

0 * " 

» - 

have been replaced with neutral phrases that fail to convey the 

speaker's attitude and manner. ^ * • 

.Information that may be critical to understanding ^the story has* , 

been deleted entirely. By eliminating the sentence about splitting 

off .natural gas In order to kill it, the explanation for the senatorj-s 

argument lfas been lost. The phrase' n by voting on it once M in the 

ft-n&l sentence of the captioned text is not a rephrascr of this missing 

► 

information; furthermore, it Is ambiguous. 

The result of these alterations fs a text that may fail to convey 
■ 

the story accurately, despite-the fact that Individual sentences are 

indeed sirrpler than the ^originals, ,Thp reader is left with a series 

• * 

of sentences that may be difficult to integrate into a meaningful 



o 
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Interpretation: thus 1 , readability may not have been enhanced* Research 

Is needed to determine whTcfi factors 4 contribute to the comprehensibM i ty 

f * ■ \ * - i * 

of, captions and to assess how we 1 1 current captioning practices fit 

the" language-processing -abilities of, the target audience. 

* , >i • Factors AffecfidJHIaption Comprehension 

Display Conventions • '* * 

In addition to deciding* what Information to include, the captioner 

must determine how" it, Is, to be displayed. The' obvious strategies are 

^not necessarily the most effective. Consider, to take just one exanple, 

- the question of how to divide ^ sentence which Is to bje captioned. 

Captions are usually limited to one or 'two 'lines so as not to 
• * % „ 

m m 

interfere with the p video. ^Because each caption appears on^he screen 

for 6nly a few seconds,, the breaks between lines on a single display 

and between successive dtsplays have more importance £ ha n they do in * 

written text, where reading occurs at an individualized pace and look- 

'backs a/e pfcssibte. In order to comprehend the captions, the reader 

f : % * 

.must ljnk the infonnatlon currently bplng presented with information 

presented earlier. Assune, for example, that the text to be captioned 

Js (l). This sentence Is £90 long to be presented on a single line. 

p t 
(I) President Carter beeves that the meeting with Reagan 

4 succeeded? * - * , ~ 

Capttoners currently ^employ two strategies In dividing such sentences 
Into lines. Either the. lines are divided So as to equalize the number of 
words per line (e.g.* 2). * 

< . * * * . 
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(2) President carter believes that the . \ # k 

- ' meeting with Reagan succeeded. * • 

or a break Is made^at a jnajpr 'clause boundary (e.g., .3). 

(3.) President Carter *bel i^ves B 
^ * that tlie meeting with Reagan succeeded. * ' 

* i . ; 

It ™ight.b£ preferable* however, to divide the sentence in a way 

that takes\into accoant some recent research which shows* that listeners 

and readerfc^use their language and general knowledge in a predictive^ 

fashion (Bobrow s Brown, 1975; Mars 1 en-Wi 1 son S Welsh, 1978; Norman s 

Bobrow, 1975). .Thus, the line division in (4) may be preferable 

because it permits th^re^der to use the word that in ,a predictive * 

fashion. t " » ' * • . 1 i 

(k) President" Carter belieyes that 
* ■ , * the meeting with Reagan succeeded. t ' 

By the end of the first line, thfe reader knows that a complement ttause ■ 



<will follow; this knowledge produces a strong expectation that facilitates 
subsequent processing even if the first line disappears before the couple- 
ment clause is encountered. Placing the complement izer at the beginning 

* ■ 

,/ .of the second line (as'in 1) requires. the reader to process back to * 

4 * 

r . p 

. ' the previous line in order % to comprehend the sentence. .This will be 
' w ^speciajly difficult if that line was on the previous display. 

• Cues to Sentence Structure * * 
In the ( service of shortening and simplifying captions, numerous , cues 

to the structure of individual sentences. are eliminated. These often 



# * 
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have the effect of snaking the sentences jpore/di ffjcult to decode. Foe 

example,* in (5), deleting the complementizer fthat still leaves a ^ 
' , . • . ■ . , 

grammatical sentence. Similarly, In (6) the relative pronoun whom 

« 

can be deleted^ 

(5) John believes [that] Harry is leaving, 

(6) The man [whom] w Sidney knows* Is a;thlef. 

• * 
% The captioned text may be shortened by deleting such cues* In the 

f 

reduced version, however, the reader must recover Information that other- 
wise would be^expli ci tly stated* Thus, -deletions ^reSMl-t in sentences 
which are shorted But which may be more difficult to comprehend. 
SJmflarly, in (7), a causal relationship between two clauses is signalled 
by .the word because. In (8), thi 9% information fs only implicitly 
conveyed in two sentences. 

♦ 

(7) Carter met P with Congressional leaders to discuss his 

- < proposed energy tax, package because^he desperately' needs 

thel r support. 

. ■ 

(8) Carter met with Congressional leaders to discuss his 
proposed energy tax package. He desperately needs 
thel r support. , * * 

* ' . ' 7. 

. It fo-ljows from these considerations that caption readability may be 
Improved by^ Adding surface cues that did not appear In the original audio. 
For example in (9), the reader must Infer that the energy bill almost 
diedj)ecause of the' battle mentioned In ^he secopd sentence. .Explicitly 



stating this Information (as in 10) may facilitate comprehension. 
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(9) The energy proposal almost died today In the, House . 

m • 

The-battle was over , , , ^ ^ 

^ (10) The energy proposal' almost died today in the House 

.because of a battle over • . . ^ 

* * * 

In otker. cases , deletions may have a posit jve effect. 

*» * • 

(II) Car la went to see* her -parents, who are living in Chicago/ 

^12) Carta went to sjee her parents. They are living in Chicago* 

Sentence (l+^-cQ^gists of 6 complex construction in whichia restrictive 

J x 

relative clause is signalled by ihe word wh£. In (l2)/this has been 
broken into two syntactically simple sentences but without req^rjjxg-an 
additional inference. Thus, (12) may bfe^sier^fofprocess. * 



obaJ-~STructure Cues 

The above examples show that comprehension of a sentence may be 
slowed or disrupted when cues to its internal structure are deleted. 

A similar problem exists the next higher 'level of analysis, the global 

* L * * 

structure .of the text. Several recent theories describe the'intemal 

structure of narrative and expository texts (e.g., Bruce S Newman, I978; 

Hallfday S Hasan, I976; Kintsch s van Dijk, .1978; Handler s Johnson, • 

* * 

1977; RumeJhart, 1975; Stein s Glenn, 1979). Cues to the Internal 

• * 

structure of a captioned te,xfare often deleted in the effort .to shorten/ 

and sirnpUiy.^ This is seen In the following example: 
• • / 

Captioned version* -! first canfe to the Middle. East II 
years ago after the six-day warr Marty Israelis thought it , * 
would be. the last war. 'This program Is not about armies or 
diplomats* It is about two families caught Jn^ thejjTddfe 
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* '* East conflict./ The program was fllmad 3 years ago. But the\ 
sam6 feelings remain jtoday as they have for the Ipst 30- 
, years- r The feelings remain even though there was hope from 
* the historic visit to isratfl by Egypt ^President Anwar Sadat- 

" This -program was the idea of an Israeli TV producer and a / 

• ft * ' 

Harvard professor-^ 

The. most striking characteristic of the text Is that^the^enrefices do not 

appear to follow from one aft^ther. 'l^T^^Xz^iV^ts frequently; a. topic 

JjitjottJurcd^a^ text (etg., information abodt the program) is 

Abruptly reintroduced after much* intervening text* The passage does 

not rea<J as^ a wel 1 -constructed story, but >ather as a series of short, , 

disjointed sentences* This may make comprehension more difficult (see 

Section 4, by Brurt S Rubin). 4 . ■ ■ 

The story lacks, structure because information that provided structure 

in the original has been delated. For example, the sentence "This 

progfram rs not about armies or diplomats" entails a sudden change of 
»*• * * * 

topic* In the original, however, it was introduced by a transition 
sentence, "VAiat you are about to see Is the first in a series about 
the Middle East-" This sentence informs the reader about the structure 
of the story and prepares him/her for elaborative information. 

• * r 

* 

Similarly, the sentence "This program was the idea of an Israel i'TV 

■* * 
' producer and q Harvard professor" also Involves an abrupt change, .of 4 

topic* Agdin, however, in the original it vfes introduced by a trpnsition 

^entence, "Before we begtrf this series, a word about how it alK carte to 

be and abbutistome of the unusual people who were involved In it/ 1 
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These examples suggest £ha,t Introductory material and expH_£it transitions 
may facMttate the conprehensloi^ppoce^ eveh £hough thdy lengthen the 



text. 



Lexical Choice , 

* * * ' 

It is important that captioners use words and expressions that are 

within the vocabularies of the .target audience. Unfortunately, precise „ 

I - * 

Information of this kind is not known. A number o/ stgdies have, shown 

y J * 

I 

that hearing-impaired persogs typically exhibit low-level reading. ski 1 Is 

i , 

(for review, stee Confad, 1979). On this basis, captioners delete or . 

replace difficult and unusual words- In addition, idiomatic expressions, 

] ** • 

\ 9 * - 

metaphors, and other faAeiful language ate deleted or replaced, 
* . i* * # * ' 

This practice presents us with three questions: First, do th§ 
! \- .> 

low-level redding , ski 1 Is of Tnany h*ar ing-impai reef p^rstfns reflect a 

deficit at the vocabul'arj* level, or are they attr i but<ahJe^to other 

factors? Se.oond, what are the capabilities of hearing-Impaired persons 

for understanding non-literal language? Third, what are *the. consequences 

^t)f altering Jhe vocabulary items or thenon literal language of a text? * 

Expressions are frequently replaced with concrete language which 

.does Jiot preserye the meaning of the original. In one/story, the idiorn 

"as^time ran out 1 ' was replaced with ."near the end," fchlch fails to convey * 

^he idea of a deadline? tn another story, 'V/orked iny way through college' 1 

was replaced with 'V/orked so \ could go to col lege, "^which "again does 

* * * 
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not capture the origtnpl meaning. In addition, these expressions are 



J- 



less interesting than the originals; replacement produces a homogeneous 

text that lacks any style and may be boring to^fcead. 

' , • $ ■ \ 

. It is also important to evaluate the effect? of idiomatic expressions 

h 

on the "comp rehensJ-ofi of the longer text in which they are embedded. If 
idioms are more difficult, they wi 1 1 1 r«qui re; t ime and processing 
resources which, would otherwise be used i-n^anaiysis of the continuing 
text. Hence, a "rl-pple' 1 effect may occur in which overall comprehension 

of the extended text suffers as weJ L Note that the opposite could 

V > 

hold instead. Idioms might disrupt processing at aTcfcal level but, 
by adding to the liveliness or character of**the tex»t, might at £he same 
time enhance its interest value for th£ readers This could lead to 
increased attent i veness , which would facilitate comprehension of *Ke 
larger- text. Furthermore, the. Iarger>«xt can*provide additional 
information which reveals the meanihg of t[ie idiom o^permlts the viewer 
to infer it A This i$sue can only be resolved through direct comparisons 
of idiom comprehension^ limtted and^extended contexts. 

Metaphors may have an even greater effect on the comprehension of 
longer texts* While ajnptaphor may increase the difficulty of the 
sentence in which it occurs, it may facilitate the comprehegfSion of 
subsequent fnformgtion by providing a^frameftork for additional detail. 
This can only be observed if the metaphof i*s embedded in a longer 
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r 1 
Genres 

Any development of general cap t4tJrfTng techniques must be done whth 
the realization that various types of programs may requl re, different 
guidelines. ^Jie experience of WGBH 'caption^rs and^prel iminary analyses 
of several captioned programs indicate tha^t all of tte .factors discussed 
above interact with the genre of the show* Far example, for Hawthorne's 

4 V 

Scarlet Letter , a prime consideration was to preserve as much as possible 
of the original language—including most of its metaphors, idiom*, and 
dialectal idiosyncrasies. In contrast, idioms and metaphors are 



C systematically deleted for news captions 



Recent research in 'text comprehension has focused on the effect of 
the perceived genre of a te*t (Brewer, 1980) on readers' expectations ' 



* and understanding. Careen (1979) demonstrated that a newspaper story 
presented as a narrative is perceived as disconnected and incompre- . 
hensible; Adams and Bruce (1981) discuss the knowledge about fables' 
which is crucial to arriving at the "standard" interpretation of Aesop's 
fables* These genre distinctions have impl ications. for captioning, 

On the most general level, ^adult programs which have been captioned 

fall into three categories—dramas, documentaries, and news. Altl)ough 

^ * • > 

the "category of dramatic presentations is hardest to describe in general 

terms, some captioo-ing issues appear to arise mokt 6ften within this* 

• * 4 4 

group. For example, because most dramas have a Targe amount of dialogue , 
placing captions so the speaker Is identifiable is*aa- issue of prime 



1]£ 



» y % r v ' *" Readability Conference 

\ • ,-; • • 

concern; of f^rtera .speakers present a difficult problem. In the 

Hi r^jC lcf Worke r * -capt i one rs soWed the problem of representing the voice 

• ■ 
of Annie SuJ H van's dea'd brother's ghost by using a smaller font, 

placing the captions a^ove Annie SuIUvan f s head and, preceding the * 

spoken w^rds with Jecnmie*s «voice. Any guidelines for display 

* * ** • • 

conventions will have to be flexible enough to a How fcf? such genre- 

Specffrc* requirements. , 

* <- * /' 

Docume/itarles usuaMy focus either on a person (e.g., Cat; about 

— 7 

a female boxer) or a situation (Arabs and Israelis); in either case, it 



el is) ; in either ca 
t the topic *nd the 



is import arifcSto identify early in the show what the topicVnd the scope 
of the discussion will be. The original version of the captioned „ . 
passage fronvArabs and Israelis quoted above performed t± is function 
fay identifying the families from whotfe perspectives the conflict will * 
be viewed, and Indicating when and where the program was filmed. Much 

* * . 

of tjifs material provided crucial organising information. Our pre- 

liminary observations indicate thqt such cues to global structure 

i - 

assume a* centra f role in documentaries. 

Hews programs may seem like the most' uniform genre. However, even t 
within a single^news show, thefre mayrbe examples of sJbgenres which 
require different captioning stylfes* We will contrast two of these 

here. The first is the Quickie" .story which occurs after the major 

» * 

news is communicated. The,' anchor person i^ads the story in lfr\5 
* %0 * 

seconds; often the last sentence is a humorous punch line, Jhe second 
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J* • 

Is the major news story, a # *l- to 2-m!nute..report, often by a correspondent 
in another city, of ten complete wl distort quotations and interviews 
and some changes in ^the accompanying^ video. 

The Quickie news story presents particular problems for captioners 
because of its brevity* The density of facts squeezed into 'such a short, 
time limits the modifications which can be made' to the text. Consider 

the following story:^ *■ 

• * 

Originals -James Earl Ray was married today in the\ 
m visitors' lounge of the Tennassee prison where he is serving * 
a 99-year sentence for the assassination of Dr. Martin* Luthur 
King, Jr« His bride is a 32-year old freelanc£ courtroom- 
artist named-Anna Sandhu. Immediately after the'ceremony Ray 
was sent back to his cell and his wif errant hon*i. * Said'flrs. 
Ray, "I would like to liye life normal ly 

c » 

The story has two main goals: ^ (a) to communicate the faqts of Ray's 
marriage and (b) to deliver a humorous, memorable punch at the end. 
Reducing. the number of words in this story will necessarily involve 
deleting some concepts whUh might be crucial to 'the total message, 
sacrificing one of these two goals! In the captioned version att of 
the facts were retained ancTthe punch Tine deleted* A different " 
decision might haye been to sacrifice the details, to keep the humor at 
the end. * « • - 

Contrast this brief news item with the energy story discussed in 
the jssues^ section. For the ntfws item tjiere is only one speaker; for 
the energy story there are two* When the video changes from the .reporter 
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to tfie senator, thfe senator's, words must appear on the screen £t the 
* • — ' * 

same time as* hfs face.. The ceptioner responded to this cpnstraint by 
i * • • * 4 

. deleting the sentence ( ,l Some senators . . . M ) which had explained the 

) » fc • ? ' * , 

senator's statement. k\ alternate strategy would have b6en to pombine. 

the first three sentences into two,, attenpt ing to retain* both the f 

introduction of topic (acconpl Fshed in the first and second original 

sentences} and the explanation of the senator's ccyrments (accompl i sn&d 

by the third sentence) * 



Sum mary . , * 1 

ThjB clear point emerging from these preliminary analyses is that / 
producing easily comprehended captions Is not simply a matter of making 
them lexically and syntactically simple., A captioned program presents 
a text^vith an internal structure dictated by the script,, Local changes 

in the text may do violence to its overall coherence. This may actually 

* * 

decrease the level of comprehension (or perhaps fail tg increase compre- 
hension in (he manner intended), ^hus; Sinple sentences rio not 

necessarily produce simple texts. An individual word, phrase, or i 

J* * * 

'feentefice may be easily, comprehended, whil^ the story in which it. Is 



embedded is not, 



i Dl scussJ^n 

The discussion begaff with several participants remarking on the 
rote of subtitles In watching foreign ;ftlms as .well as. on television in 
other, nbn-Engl Ijh-spfcaking countries. " 1 * 

* 

v ^ ; ' . \22 * * ; ^ V 
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• Sgklen : Somfe learning can take place just by observing 'the video 
portion of foreign films and matching itwtt'h the, text/ t 
, . Selde?fberq : Viewers would appear to' have some kind of 'toetaknowledqe 

* <: . 

of how captions typically relate to the pictures being shown-"- Such -* 

ft * w 

knowledg^rmay pean that the amount of redundancy now In evidence ajuJd 
bfe reduced, v Of coyrse there remains the question of the composition 
of the target population and whether or not such a population has these 
skills^* The net effect of d£tetl*ons seem$ to tfe to force the perceiver j 

X f 

to generate inforrait ion -which was in the original text. J^t is, by 

shortening the text, very little "is "really to be gainjd if the reader 

then Iras' to gener^j^nore inferences? to make. sense of the text. 

Kantor r Perhaps t work. needs to be*done to' determine genre differences 

between texts. There is obviously a difference between the flowing ' 

information of the ABC Eyeping News' dnd a drama such as The Scarlet ♦ 

* * ~ " — — * 

letter. The c&ptipners of the latter broadcast seernpd to have* little 

trouble keeping "the captions verbatim representations, of the audio, 

/ J m + . 
principally because the *Lrf1osfue was. brief . 

* i * 

• ■ ■ * 

Seldenberq: Gaptloqers are clearly aware of such genre differences. 

flubln ;. In fadt, there was actually a policy decision at the Caption 

Center to change as Mttle of. the language of Trte Scarlet letter as . 

- ; > - ~ ! : 

possible because ofjthe Important- ^ftfect of ^style In such a work, 
(Incidentally, $hey also captioned a production of The Miracle Worker, 
resulting,!/) some rather circular effects.) the hardest material to 
delete occurs^ln so-called "Peorfa sEo?Te$ ,r (e.g. , "There was a fire/ 

} * ' *" ' 
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this afternoon In Peoria, Illinois, and x ntrober of people died . v 



T 

srr^e they contain Ifttle tf any deTe^le 
more than ?5 seconds to report- 'The qaptiinlng simply is not /able to 



keep up with the audio. 



» * 



Information and may take po 



. Seldenberg : And of course, there is the political decision oever to 



eliminate entirely any one npws item- * / 

* * * 

Griff inr : S^ne research has ifeen conducted on little children 

B r 

watching television programs that were beyond their level of comprehension. 

The initial episode 6f> Superman , for example, in which the*baby|/is 

■ * ' •/ \ 

launched in a rocket to escape the destruction of the planet Ktfypton, * 



4 * 



/- 



is extremely .confusing.* A roomful of young children watching,, that kind^ . , 
of prograri reportedly talked' throughout it, asking each othef questions . 

and helping each other understand. When some caption readers watch 

/ 

television K they,make It a practice to form groups, making'the viewing 
a social event". It may be that social collaboration amprv£ caption 



Viewer^ 



as was 



can do some of the work in trying to understand ,jthe program, 



the case wittf the chl ldren t vlevprs* Wille suchr'inUraction qannot 

i ' i 

be qssuped, it m&y be worth investigating hdw f r^querjt ,lt^ is and how ^ 

i • . t ^ 

it effects caption viewing., * • $ £ 

Kantor; Since cable television may someday enable individual caption 



viewers to choose the level and rate of captions by providing one channel 

, ■ i • F 

for .verbat-Im captioning, one for very simplified captions, and so on, 
future policy decisions may not have to deal with the problem of "targfet 



audience, 1 * as is currently the case. 
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• Sgl den berg : Hdwever, Tt Is doubtful that the commercial networks 
wi ll^adapt j*rogFammlng to sulfa minority of viewers who rely on 
captions* It i* really something of a miracle that television Is 
going to gain In importance among this group, but there is a real problem 
wi^h the way in which captioning is being Implemented- There is^also 

9 a problem with' the,**soluiJpn of the new decoded captions- Apparently * 

f , « 
the quality of resolution is very poor and it is difficult to read 

the 'captions. t ^ 

( k - 8.' Final Discussion * 

The final session of the Conference took the form of an open dis- 

0 

cussion, which centered around what were felt to be the two main issues - 

• 1 * * * 

at hand: (a) What are the legitimate arguments against the use of 
readability formulas for certain purposes? and (b) What aVe sound 

alternatives to readability formulas, and how can they be supported r 

» 

by enpirical evidence? 



Arguments Against Some Uses of Readability Formulas - ' 

Readability formulas have been Immensely poplar with publishers and 
* 

some educators, ,desp,lte their limitations. Given tha't there are some 
problems with the formulas as they r\pw exist \ the opponent^' of the 
formulas have to deal with why^they remain so popular ancl how to render 
them lesS so. 



9 

ERLC 



1-25 



/- 



Readability Conference 



' - i ' ' 119 



Green : One possible jway of dempnstratlng the problems -wi th formulas \^ 
- ' * \ " \ ^ i * 

is through field testing; fielcLtests might show that material which 

} 

scores poorly on readability formula's Is, nevertheless readable. 

Selden : jSuch' studied have beervdone in <pnneGtion wl2h tax forms, 

V ' . y ■ , ^ • . . 

J)ut one cannot inmed lately generaffi^e to children results that are 



obtaihed with aduifs, or vice versa/ Studies specifically designed to 

challenge the use bf formulas on children's texts by showing that some 

< i 
texts which score well do J\ot 'V*)rk H ds well as texts which score poorly, 



probably should u be*. done. A variety of tasks could be 'used to measures . 

how well children understand a text^ It is imperative that the specific 

task be defiried, before any test of rea<fabi 1 i\ty is devised, and the task 
~~~~ ** , 

must be atj appropriate one. Cloze tests, for example, are not particularly 
natural tasks'for children. * , * 

Kantor: Individual studies might be aimed at showing that one 

* - ~ * 

particular effect of th$ application of readability formulas* is bad, 

e.g. , sentence. spl i tting. 1 t 

* •'• 

Selden: But it is clear, however, that formulas are not intended 

to be'used as*guides foV wfitingi sB this type of criticism would only 1 

apply to Improper uses of the formulas. < » 

Bruce: The burden of proof may well lie with the proponents of the 
■' — - < / 

formulas. To what extent are the formulas better predictors than the 
average School teacher or librarian? 

Clyrrow t /The formulas have' been reylsed a number of ttmes, presumably ^ 

> • * 

to make them.more and more accurate. 
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Selden: But validation studies have been unbelievably weak; 
validating one formula against another. Furthermore, the results have . 
been generalized far beyond the initial texts. • * 

Griffin: Basic to the use of readability formulas has been the 

/ . 

notion jhat statistics, numbers, carry more truth than do other kinds 

of facts. The use of reading, levels relies on a kind of generalization 

that if someone can read at a given level, e.g.. ,.6.2, then he or she can 

read 'anything that measures bejow that level. Tfrts generalization is . - 

unwarranted because the formulas are very text specific. 

« Bruce: The effect of cultural backgr&unds is further evidence that 
• ■ * 

the generalization of reading levels is not quite right. On. such tests 
•as the BEAT an& 8ITCH,.for example, blacks outperform whites. This 
points to, factors such as culture which Influence the "readability" of . 
a text 4 but which are necessarily ignored By the formulas. 

Seldenberg; The pressures for the use of readily applicable formulas 
appear, to supercede, all *\hese considerations. It Is obvious that, given 
the needs of people who are unde^ state mandates to use formulas, 
formulas are simply a fact of life. # It Is another issue, however, to 

* * 

de£e/mlrte whether techniques are necessary which permit us to predict 
ahead of time how[ readable a text Is. Good writing, apparently the . 

? * , / ' V 

result of talent, Is clearly beyond the predlctlve / realms of the formulas. 
Likewise, the ethics of whether or not an adaptor has the right to -tamper) 

. * • 

with the content of a .text—deleting Information, for exanple—has not 
been addressed by the proponents^ formulas. \ 



* i 
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Davison: \{t Is probably af mistake to mix the kitties and economics 
of readability formulas with the legitimate logical arguments against them* 
^ I Charrcw : The politics and ecorromlcs ye In their favor, particularly 
slhce the powers that be (e.g., legislators) really believe that formulas 
are an accurate gauge of comprehension. 

Davison : The most we can hope to accomplish is to spell out a 
systematic attack based on the logical arguments against readability 
formulas in theJr present state, With such 3 list In hand, we could then 
go about the business of convincing others that there 1$ £ strong case 
to be made (against the Indiscriminate Use of formulas.. That, at least, 
would b"e a 'beginning. v 



4 



f The major logical arguments against readability formulas are as 

* . ** 
follows: 

• • • * *- j 

1. Morgan (Linguistics, Unlversjty of Illinois): The validation 

% studies have Been done are both»weak and circular In nature. What little 
'data there Is tends to validate one* formula against another. 

1 

2. Morgan*: The mistake Is made of particularizing from the average 

— • i - i • 

t6 a single sentence--! .e^, from mass statistics to one. text and one reader. 

It Is assumed that slnc^ mass , statist Ics shbw-V general tendency, one can 

• " I 

particularize to one passage for one child. Th?s Is a non-sequl tur. * , 

* 4 * ' / 

3« Bruce : Related to 2 Is thef questionable practfee of general Ijsjng 

* I 

or extrapolating from the original passage upon which the readability formula 

4$ based to any passage. It Is poon statistical method to extrapolate to 

* m % 

another population from that of the ^original study. 
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4. Morgan : The parameters which are measured by readability formal as 

p are Incomplete. There arte numerous factors which go unmeasured by the. 

formulas but which contribute to readability. These Include specific 

syntactic characteristics, pragmatic considerations (e.g.,. inferences),* • 

"ciHtural aspects, how enjoyable a text is to read, and so forth. V 

5- Morgan : It is probably unreal is tic *to expect any formula to deal ( 
* • 
with all of the factors which affect readability. A simple mechanical* 

measure will be 1 limited to certain characteristics of a text, therefore, 

no readability formula should be regarded as adequate for measuring 

readability. . r ' 

, 6. Sejden : Related to (5) is the caveat that if a fofmuIaNs 
— * * * 

designed to measure a given task, A, on a given group, B, it should only 
„b£ tised for that. Because the foiynulas are so widely believed in, there 
is a .tendenqj/to misuse "them. Included *n the misapplication, of readability 
formulas is their u§e as guides to writing or adapting, acknowledged as 

* ■ 

a misapplication by the designers of the formulas. 1 

Much of the discussion centered arouod^ and the no^on that 

certain parameters could not be included In formula^, at least not without 

making them Into cumbersome and hopelessly complicated mechanisms. What* 
« ■ 

ever "gains would be made by attempting to include such factors would be 
vitiated by the dl ff Ic ujt les^whlch (Che new formulas would create. (They* r ; 
would cease to be quick and easy.) * -* 
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Davison ; What* becomes increasingly clear, however, Is just how ' 
^ * * * 

imprecise the formulas are at present* The notion of grade levels dates * 

£ack to the era.o/ the McGuffy ReadefS^ The levels are now taken for 

granted to be meaningful, and readability formulas measure texts against 

them. Just how absurd this notion is should be evident when the formulas 

are used to determine the j'grad'e level" o^fa tax form Intended for ^adults 

who are* required to perform a specific task (unrelated toschooT texts 

* M 
fc * 

in both m form and purpose)* Jn fact, it may on occasion be more difficult' 

M * - 

to read at* a lower level than one Is used to, since much of the informa- 
tion from which to draw inferences wojj]d be lacking and the reader 

^ould need to dp Trore'work in order to extract the same amount of 

* 

information. • m ^ 

A. Zwicky, Columbus, Ohio ; While it is clearly not advisable to 

Idi > 



give children passages whl< 



f standpoint <5f^he fonrtulas, 



ar.e much too difficult for thenr, some 



challenge, at least, might be%eneflclal and help improve the child's 
reading ability. Furthermore, despite improvements In texts from the 

* if \* 



there is little' evidence that reading 



scores have been improving ct/er the years. , Reading scores may, in 

fact, be deteriorating, though .this is difficult to determine due to h 

f . 
changes In -language, population, and culture over the years • 

• Alternatives to Readability Formulas 

» Bruce : For all of the.reasorts mentioned above, mafcy people do feel 
a need for some standard of measuring how readable texts are* One' 
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<obvIo'u? alternative to formuJs Is to flerld-test materials- By giving 

i 

a text to people and actoaUy^seeing whether the text causes compre- 
hension problems, we dan achieve results which will probably be^as 
good as, if not better than, the information provided by the standard 

readability formulas. * 
* * • 

Davison: The formulas are now beln§ used to handle such v texts as 

contracts and tax forms ^-designed, of course, for adult readers—and 

it- is imperative that the content of these texts be easily understood 

* - * * 

by the readers. Since 10 studying these we would be dealing with a 

relatively small corpus of texts (unlike the situation with basal 
readers or trade books for children), wi th a relatively closed set of 
instructions for the reader, field testing seems both appropriate 
and feasible. * - * 

Selden : Publishers currently spend large sums o£ money to have 
programs analyzed by readability formulas. That same mtf^y could be 
put to better use by having their new texts field tested. 

TIerney: Currently, materials designed for the classroom must be , 

♦ * \ 

approved of by teachers, and thfrs itself is a kind of Field test- Field 

* *■ * 

testing was In fact.done In the Natural Assessment program by Petroski 

# * ■ 

(Personalfcorrounication) at the UniversU/ of Pi ttsbyrjjh Fn conjurfction 
wUh^schobl librarians. * Classes^of -9- and 13-year-olds were allowed to 
make comments on readings and asked to eliminate passages which they dis- 
liked/ The criteria were how enjoyable a text was and how well suited 

\ 

it was to various purposes. 
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^ ^ Morgan : Of all the skills taught to children, pnly reading Is sub- 

' ^ * \ ,' t . . 

ject to the supposedly objective method of the readability formulas. , 

<-> - 

GrtffuK "This is in part due to the hierarchical nature of such 

* \ 

areas as mathematics, wher$ the basic skills of addition and subtraction 
are considered to be esscyitlal to the acquisition of more advanced . 

• y - • 

skills such. as multiplication and division. . ' 

* * * ■ 

Selden : But there is also a sense that .readabJJ i ty fof mula^s have- 
gotten out of hand, since they were originally no more than an attenpt 
to control the vocabulary in basal readers, ^*g., Thorndike's word U^t , 
for use^ in vocabulary recognition Skills. 5 

. Char row : Parallel 'to the development 6f math skills, there has been 
an attempt to build a hierarchy into the reading process. ,^ • " 

Tlerney ; Teacheijs are told that the formulas* are used as a ya*d- 

stick which, on an analogy with a shoe- store, will give each child a f 

custom fit. Jhe application of Harris 1 fjve~finger rule— i.e., more 

* • 

than five worlds per page which cause the child problems means that the 
text is inappropriate-H^pqrt of this Idea of custom fit. 

Kantorr By field testing, it would be possible to determine just 
whdt syntactic structures, cause problems fof* the target readers. 

Osborn : This Would certainly have a high degree of validity and * 
wouid relate to what the readers actually w6re able to handle. 

Tjegney : But agalrt, we are faced with; the pragmati'c consideration 
fat thi^mighttnot be the "quick and dirty?' method Whi< 
want. 



9 

ERLC 



, M Readability Conference 

* *- 

, - ' ■ 126 

* * / 0 9 * 

Griffin: Still ^ It ought to be possible to'ldd on some 6f these 

higher-level" skills tp the a lready / exi st ing hierarchy. Whether or 

ndt the hierarchy' idea itself Is defendable has been the subject of ^ 

Ruch controversy. 

. Tiemey : Publishers are being forced to homogenize texts— I.e., 
level out/the difficulties and make all the passages conform to an 
average. mm 

. Day i son : Such a trend aeems both pernicious and unnecessary. 
Selden : Regulating language in this way provides such groups as 
insurance companies with numbers which "prove'J that a form conforms to 
tertain averages. This i^s an easy^ w^y \o reply to challenges which 

might arise' that a form is, for instance, too difficult). Similarly, 

* * * \ • • 

school boards can point to the formulas vt)en challenged that their x 

% \ * • 

reading programs .have failed. Formulas are called upon despite the 

discrepancies which exist between the scores according to the various 

* , * > / m * 

formulas, ,The fojmu^as look^ objective, people think that they are 

^ v, : 1 " 

measur Ing ~£orne taring 6f Importance, and they a re, quick and easy to apply. 

Thus, the formulas remain Incredibly attractive to many, even when mjs- 



. applied tp rewrite texts (e.g., the case of the IRS splitting long 
•r 

t seqterfces. even, when this resulted in instructions which were less cleai\ 



than the original). Conrnunicabil i ty seems to be less of a priority than 
'meeting sortte standard of "rVadabJ 1 1ty»" o 
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Tlerney: One concrete step which jieeds to be taken Is the djssemina- 

O 

tlon of such research as the technical report on adaptation written 

» 

by the Text Analysis Group at the Center for the Study of Reading 
(Davison et all, 1980), * h 

0sborn_; That particular report Is qJlte lengthy, and It would be 
possible to extract some of the suggestions it contains and send the 
short report to publishers and\>thers In an attempt to "raise their 
consciousness" and maybe eventually meet with some of th$n and discuss 
the issues. 

Bruce: There is also the need for studies which conpare the results 

of readability formulas with those % of ^actual field testing* It w9uJd be 

possible, for exampje., to devise an experiment which dbmpared the 

"reading level of certain trade books for children with actual field 

'-tests. v 

Selden ; It would first be necessary to define the kind of task 

whit:h Is being investigated. This would determine how appropriate the 

* • * * 

text is for a given task only* 

i Osborn: Basal readers* hqtaver*' are felt by many at this conference 

tp be the primary target of bur attack. 

Char row : In terms of the rewriting of texts, It Is very possible 

to use general guidelines TnstQdd bf the rigid formulas* i * 

Bruce; For example, avoid certain constructions under certain 
\ * ^ 

condition? (not ynllke Strunk ZT White) . • 
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Davison : £RA already uses the guidelines of Dawkins's (1973) 
compendium, though the compendium really doesn't explain tfhy cer/aln ^ 
things are complex. What seems to be needed are more research-based 

guideline^ - - ' 

I * 

In conclusion, .there l^pear^o be a number of ways to determine the 

suitability of any gtven text besides the application of readability 

formulas. Guidelines can be drawn up which list principles that should 

be applied when wri ting" texts for g^ven .purposes and r«defs. FieJd 

testing can be dontiL,to determine the level of difficulty of an 
* ■ - 

existing text, its appropriateness, for well-defined tasks and audiences, 

— 1 > * * 

■ * 

how enjoyable it is, and even what could be done to iniprove it. Teachers 

■ v ■ 

r %, 

should be consulted; their insight is valuable and sterns from actual 
use of reading materials in classrooms. Children themselves are able 



to provide useful judgments. Syntactic and other parameters could 

be incorporated jnto the already existing hierarchies, thus making it 

possible to better select appropriate texts. In the meant i me K our task 

is to make available to others the logical arguments which we' have 
« * ■ 

presented here againdt the use of readability formulas. Workshops are 
needed for both teachers £nd publishers, 
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Footnotes 



I r 
J The content of thls^per does, not necessarily reflect official 

NIE policy. \ ' ^ / r 

Credits beyond specific citations are due to Cissy Freeman and * 

Sean Walmsley for both ideas and arguments. < 

3 0 * 

Front "Arabs and Israelis: Two Families, 11 a WGBH-TV production/ 

April 13, 1978. . 

• • < ' 
For additional information about 'captioning, see Section 7, 

Comprehension of Captioned Television, which was 'presented at the 
Cbnference by Marrk^Sel denberg and co-authored with B. (. Bruce and 
Andee Rubin. • 

^The \7 guidelines are listed In fcxtapent Design (197£). A revised 

edition is qow in press. ^ 1 * 

6 y % 

This is j>art of a preliminary report 'written by "Hark "Seldenbeng,* 

Bertram C. Bruce, ancJ Andee Hubin at Bolt, Beranek and Newman, Inc., 

Cambridge, Mass. More reports are forthcoming. 'Thanks to the Caption 

Center, WGBH-TV Boston, for making aval J able captioned materials and ^ 

for discussing thpse issues with us. ' * ' , ' 

7 

■ „ 'From the Captioned ABC News, October 13, 1978. 

' 8 * 
From "Arabs and .Israei isj^ Two Families, 11 a WGBH-TV production, 

*>rU 13, 1978. ' , „ 

9 ' " 

•'From the Captioned ABC News, October 13, 1978. 
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Table 1 

Comparison of Reading Levels 
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Average Number of Word£ Per Sentence 
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Dayton 


Trees 


Original 
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Table 3 



s Comparison of Word and Sentence^pngth , * * 





Call fornia 
Giants 


«s 


Disaster in 
Dayton 


Hi lk 


Living 
Light 


*/ 

Number of Words: 
* Original 


3,000* 


# 


**,725 


* * 

' 1,256 




Adaptation 


775 




827 


861 


900 


Number of Sentences: 






# 






Original 
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250* 




3*5* - 


58 


75. 


Adaptation 


62 
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63 
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f • . Table A 

Clause Complexity In LIGHT, and MILK 



; f • - 

Number of Sentences In! 



''Light. 




Number of Clauses Per Sentence 

' T' # 2 ' 3 * '5 "6+ 



Original 
Adapted 



Milk 



21 . 22 .. 23 

22 20 12 



V 



Original 
Adapted ^" 



20 * 16 \ 12 '2 
29 ■ 21 15 6 



Note- In countin§^nulober of clauses , we have Included subject , object * 
^and adverbial Complement clauses, apd relative clauses, .Including 
w reduced post -'nominal modifiers and parentheticals. We have 
excluded prenominal modtflers, normalizations, and conjoined 
noun phrases t even though tfiese might have been derived frorn 
some more complex source which Involved clauses. Our criteH^h 
was,essentjal ly whethet there Is a mjjor &dnst>tuent break In 
surface structure. ,Tftough we hawPbeen arbitrary in a few cases, » 
" the measure^we give, Is, In general, consistent. 
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p Figure 1. The -Process Hodel of Document -Design. 
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